進入 sqoop2-shell
[hadoop@netkiller ~]$ sqoop2-shell Setting conf dir: /srv/apache-sqoop/bin/../conf Sqoop home directory: /srv/apache-sqoop Sqoop Shell: Type 'help' or '\h' for help. sqoop:000>
Sqoop client script:
sqoop2-shell /path/to/your/script.sqoop
sqoop:000> show version client version: Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
sqoop:000> show version --all client version: Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016 0 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable server version: Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016 API versions: [v1]
sqoop:000> set server --host master --port 12000 --webapp sqoop Server is set successfully
sqoop:000> show connector 0 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable +------------------------+---------+------------------------------------------------------------+----------------------+ | Name | Version | Class | Supported Directions | +------------------------+---------+------------------------------------------------------------+----------------------+ | generic-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO | | kite-connector | 1.99.7 | org.apache.sqoop.connector.kite.KiteConnector | FROM/TO | | oracle-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO | | ftp-connector | 1.99.7 | org.apache.sqoop.connector.ftp.FtpConnector | TO | | hdfs-connector | 1.99.7 | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO | | kafka-connector | 1.99.7 | org.apache.sqoop.connector.kafka.KafkaConnector | TO | | sftp-connector | 1.99.7 | org.apache.sqoop.connector.sftp.SftpConnector | TO | +------------------------+---------+------------------------------------------------------------+----------------------+ sqoop:000> sqoop list-databases --connect jdbc:mysql://192.168.1.1:3306/ --username root --password 123456
sqoop:000> show connector --all 7 connector(s) to show: Connector with Name: generic-jdbc-connector Class: 1.99.7 Version: FROM/TO Supported Directions {4} link config 1: Name: linkConfig Label: Database connection Help: Contains configuration that is required to establish connection with your database server. Input 1: Name: linkConfig.jdbcDriver Label: Driver class Help: Fully qualified class name of the JDBC driver that will be used for establishing this connection. Check documentation for instructions how to make the driver's jar files available to Sqoop 2 server. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 128 Input 2: Name: linkConfig.connectionString Label: Connection String Help: JDBC connection string associated with your database server. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 3: Name: linkConfig.username Label: Username Help: Username to be used for connection to the database server. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 40 Input 4: Name: linkConfig.password Label: Password Help: Password to be used for connection to the database server. Type: STRING Sensitive: true Editable By: ANY Overrides: Size: 40 Input 5: Name: linkConfig.fetchSize Label: Fetch Size Help: Optional hint specifying requested JDBC fetch size. Type: INTEGER Sensitive: false Editable By: ANY Overrides: Input 6: Name: linkConfig.jdbcProperties Label: Connection Properties Help: Key-value pairs that should be passed down to JDBC driver when establishing connection. Type: MAP Sensitive: false Editable By: ANY Overrides: link config 2: Name: dialect Label: SQL Dialect Help: Database dialect that should be used for generated queries. Input 1: Name: dialect.identifierEnclose Label: Identifier enclose Help: Character(s) that should be used to enclose table name, schema or column names. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 5 FROM Job config 1: Name: fromJobConfig Label: Database source Help: Specifies source and way how the data should be fetched from source database. Input 1: Name: fromJobConfig.schemaName Label: Schema name Help: Schema name if the table is not stored in default schema. Note: Not all database systems understands the concept of schema. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 50 Input 2: Name: fromJobConfig.tableName Label: Table name Help: Input table name from from which data will be retrieved. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 50 Input 3: Name: fromJobConfig.sql Label: SQL statement Help: Import data from given query's results set rather then static table. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 4: Name: fromJobConfig.columnList Label: Column names Help: Subset of columns that should be retrieved from source table. Type: LIST Sensitive: false Editable By: ANY Overrides: Input 5: Name: fromJobConfig.partitionColumn Label: Partition column Help: Input column that should be use to split the import into independent parallel processes. This column will be used in condition of generated queries. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 50 Input 6: Name: fromJobConfig.allowNullValueInPartitionColumn Label: Partition column nullable Help: Set true if partition column can contain NULL value. Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 7: Name: fromJobConfig.boundaryQuery Label: Boundary query Help: Customize query to retrieve minimal and maximal value of partition column. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 50 FROM Job config 2: Name: incrementalRead Label: Incremental read Help: Configures optional incremental read from the database where source data are changing over time and only new changes need to be re-imported. Input 1: Name: incrementalRead.checkColumn Label: Check column Help: Column that is checked during incremental read for new values. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 50 Input 2: Name: incrementalRead.lastValue Label: Last value Help: Last imported value, job will read only newer values. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: -1 TO Job config 1: Name: toJobConfig Label: Database target Help: Describes target destination and way how data should be persisted on the RDBMS system. Input 1: Name: toJobConfig.schemaName Label: Schema name Help: Schema name if the table is not stored in default schema. Note: Not all database systems understands the concept of schema. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 50 Input 2: Name: toJobConfig.tableName Label: Table name Help: Destination table name to store transfer results. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 3: Name: toJobConfig.columnList Label: Column names Help: Subset of columns that will will be written to. Omitted columns have to either allow NULL values or have defined default value. Type: LIST Sensitive: false Editable By: ANY Overrides: Input 4: Name: toJobConfig.stageTableName Label: Staging table Help: Name of table with same structure as final table that should be used as a staging destination. Data will be directly written to final table if no staging table is specified. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 5: Name: toJobConfig.shouldClearStageTable Label: Clear stage table Help: If set to true, staging table will be wiped out upon job start. Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Connector with Name: kite-connector Class: 1.99.7 Version: FROM/TO Supported Directions {4} link config 1: Name: linkConfig Label: Global configuration Help: Global configuration options that will be used for both from and to sides. Input 1: Name: linkConfig.authority Label: HDFS host and port Help: Optional to override HDFS file system location. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 2: Name: linkConfig.confDir Label: Hadoop conf directory Help: Directory with Hadoop configuration files. This directory will be added to the classpath. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 FROM Job config 1: Name: fromJobConfig Label: Source configuration Help: Configuration options relevant to source dataset. Input 1: Name: fromJobConfig.uri Label: dataset:hdfs://namespace/table Help: Kite Dataset URI from which data will be read. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 TO Job config 1: Name: toJobConfig Label: Target configuration Help: Configuration options relevant to target dataset. Input 1: Name: toJobConfig.uri Label: Dataset URI Help: Kite Dataset URI where should be data written to. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 2: Name: toJobConfig.fileFormat Label: File format Help: Storage format that should be used when creating new dataset. Type: ENUM Sensitive: false Editable By: ANY Overrides: Possible values: CSV,AVRO,PARQUET Connector with Name: oracle-jdbc-connector Class: 1.99.7 Version: FROM/TO Supported Directions {4} link config 1: Name: connectionConfig Label: Oracle connection configuration Help: You must supply the information requested in order to create an Oracle connection object. Input 1: Name: connectionConfig.connectionString Label: JDBC connection string Help: Enter the value of JDBC connection string to be used by this connector for creating Oracle connections. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 128 Input 2: Name: connectionConfig.username Label: Username Help: Enter the username to be used for connecting to the database. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 40 Input 3: Name: connectionConfig.password Label: Password Help: Enter the password to be used for connecting to the database. Type: STRING Sensitive: true Editable By: ANY Overrides: Size: 40 Input 4: Name: connectionConfig.jdbcProperties Label: JDBC connection properties Help: Enter any JDBC properties that should be supplied during the creation of connection. Type: MAP Sensitive: false Editable By: ANY Overrides: Input 5: Name: connectionConfig.timeZone Label: Session time zone Help: timeZone Type: STRING Sensitive: false Editable By: ANY Overrides: Size: -1 Input 6: Name: connectionConfig.actionName Label: Session action name Help: actionName Type: STRING Sensitive: false Editable By: ANY Overrides: Size: -1 Input 7: Name: connectionConfig.fetchSize Label: JDBC fetch size Help: fetchSize Type: INTEGER Sensitive: false Editable By: ANY Overrides: Input 8: Name: connectionConfig.initializationStatements Label: Session initialization statements Help: initializationStatements Type: LIST Sensitive: false Editable By: ANY Overrides: Input 9: Name: connectionConfig.jdbcUrlVerbatim Label: Use JDBC connection string verbatim Help: jdbcUrlVerbatim Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 10: Name: connectionConfig.racServiceName Label: RAC service name Help: racServiceName Type: STRING Sensitive: false Editable By: ANY Overrides: Size: -1 FROM Job config 1: Name: fromJobConfig Label: From Oracle configuration Help: You must supply the information requested in order to create the FROM part of the job object. Input 1: Name: fromJobConfig.tableName Label: Table name Help: tableName Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 2: Name: fromJobConfig.columns Label: Columns Help: Columns Type: LIST Sensitive: false Editable By: ANY Overrides: Input 3: Name: fromJobConfig.consistentRead Label: Consistent read Help: consistentRead Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 4: Name: fromJobConfig.consistentReadScn Label: Consistent read SCN Help: consistentReadScn Type: LONG Sensitive: false Editable By: ANY Overrides: Input 5: Name: fromJobConfig.partitionList Label: Partitions Help: partitionList Type: LIST Sensitive: false Editable By: ANY Overrides: Input 6: Name: fromJobConfig.dataChunkMethod Label: Data chunk method Help: dataChunkMethod Type: ENUM Sensitive: false Editable By: ANY Overrides: Possible values: ROWID,PARTITION Input 7: Name: fromJobConfig.dataChunkAllocationMethod Label: Data chunk allocation method Help: dataChunkAllocationMethod Type: ENUM Sensitive: false Editable By: ANY Overrides: Possible values: ROUNDROBIN,SEQUENTIAL,RANDOM Input 8: Name: fromJobConfig.whereClauseLocation Label: Where clause location Help: whereClauseLocation Type: ENUM Sensitive: false Editable By: ANY Overrides: Possible values: SUBSPLIT,SPLIT Input 9: Name: fromJobConfig.omitLobColumns Label: Omit LOB columns Help: omitLobColumns Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 10: Name: fromJobConfig.queryHint Label: Query hint Help: queryHint Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 11: Name: fromJobConfig.conditions Label: Conditions Help: conditions Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 TO Job config 1: Name: toJobConfig Label: To database configuration Help: You must supply the information requested in order to create the TO part of the job object. Input 1: Name: toJobConfig.tableName Label: Table name Help: Table name to write data into Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 2: Name: toJobConfig.columns Label: Columns Help: Columns Type: LIST Sensitive: false Editable By: ANY Overrides: Input 3: Name: toJobConfig.templateTable Label: Template table name Help: templateTable Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 4: Name: toJobConfig.partitioned Label: Partitioned Help: partitioned Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 5: Name: toJobConfig.nologging Label: Nologging Help: nologging Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 6: Name: toJobConfig.updateKey Label: Update key columns Help: updateKey Type: LIST Sensitive: false Editable By: ANY Overrides: Input 7: Name: toJobConfig.updateMerge Label: Merge updates Help: updateMerge Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 8: Name: toJobConfig.dropTableIfExists Label: Drop table if exists Help: dropTableIfExists Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 9: Name: toJobConfig.storageClause Label: Template table storage clause Help: storageClause Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 10: Name: toJobConfig.temporaryStorageClause Label: Temporary table storage clause Help: temporaryStorageClause Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 2000 Input 11: Name: toJobConfig.appendValuesHint Label: Append values hint usage Help: appendValuesHint Type: ENUM Sensitive: false Editable By: ANY Overrides: Possible values: AUTO,ON,OFF Input 12: Name: toJobConfig.parallel Label: Parallel Help: parallel Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Connector with Name: ftp-connector Class: 1.99.7 Version: TO Supported Directions {4} link config 1: Name: linkConfig Label: FTP Server configuration Help: Parameters required to connect to an FTP server. Input 1: Name: linkConfig.server Label: Hostname Help: Hostname for the FTP server. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 256 Input 2: Name: linkConfig.port Label: Port Help: Port for the FTP server. Connector will use 21 if omitted. Type: INTEGER Sensitive: false Editable By: ANY Overrides: Input 3: Name: linkConfig.username Label: Username Help: Username that will be used to authenticate connection to the FTP Server. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 256 Input 4: Name: linkConfig.password Label: Password Help: Password that will be used to authenticate connection to the FTP Server. Type: STRING Sensitive: true Editable By: ANY Overrides: Size: 256 TO Job config 1: Name: toJobConfig Label: Output configuration Help: Parameters required to store data on the FTP server. Input 1: Name: toJobConfig.outputDirectory Label: Output directory Help: Directory on the FTP server to write data to. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 260 Connector with Name: hdfs-connector Class: 1.99.7 Version: FROM/TO Supported Directions {4} link config 1: Name: linkConfig Label: HDFS cluster Help: Contains configuration required to connect to your HDFS cluster. Input 1: Name: linkConfig.uri Label: URI Help: Namenode URI for your cluster. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 2: Name: linkConfig.confDir Label: Conf directory Help: Directory on Sqoop server machine with hdfs configuration files (hdfs-site.xml, ...). This connector will load all files ending with -site.xml. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 3: Name: linkConfig.configOverrides Label: Additional configs: Help: Additional configuration that will be set on HDFS Configuration object, possibly overriding any keys loaded from configuration files. Type: MAP Sensitive: false Editable By: ANY Overrides: FROM Job config 1: Name: fromJobConfig Label: Input configuration Help: Specifies information required to get data from HDFS. Input 1: Name: fromJobConfig.inputDirectory Label: Input directory Help: Input directory containing files that should be transferred. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 2: Name: fromJobConfig.overrideNullValue Label: Override null value Help: If set to true, then the null value will be overridden with the value set in Null value. Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 3: Name: fromJobConfig.nullValue Label: Null value Help: For file formats that doesn't have native representation of NULL (as for example text file) use this particular string to decode NULL value. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 FROM Job config 2: Name: incremental Label: Incremental import Help: Information relevant for incremental reading from HDFS. Input 1: Name: incremental.incrementalType Label: Incremental type Help: Type of incremental import. Type: ENUM Sensitive: false Editable By: ANY Overrides: Possible values: NONE,NEW_FILES Input 2: Name: incremental.lastImportedDate Label: Last imported date Help: Datetime stamp of last read file. Next job execution will read only files that have been created after this point in time. Type: DATETIME Sensitive: false Editable By: ANY Overrides: TO Job config 1: Name: toJobConfig Label: Target configuration Help: Configuration describing where and how the resulting data should be stored. Input 1: Name: toJobConfig.overrideNullValue Label: Override null value Help: If set to true, then the null value will be overridden with the value set in Null value. Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Input 2: Name: toJobConfig.nullValue Label: Null value Help: For file formats that doesn't have native representation of NULL (as for example text file) use this particular string to encode NULL value. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 3: Name: toJobConfig.outputFormat Label: File format Help: File format that should be used for transferred data. Type: ENUM Sensitive: false Editable By: ANY Overrides: Possible values: TEXT_FILE,SEQUENCE_FILE,PARQUET_FILE Input 4: Name: toJobConfig.compression Label: Compression codec Help: Compression codec that should be use to compress transferred data. Type: ENUM Sensitive: false Editable By: ANY Overrides: Possible values: NONE,DEFAULT,DEFLATE,GZIP,BZIP2,LZO,LZ4,SNAPPY,CUSTOM Input 5: Name: toJobConfig.customCompression Label: Custom codec Help: Fully qualified class name with Hadoop codec implementation that should be used if none of the build-in options are suitable. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 6: Name: toJobConfig.outputDirectory Label: Output directory Help: HDFS directory where transferred data will be written to. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 7: Name: toJobConfig.appendMode Label: Append mode Help: If set to false, job will fail if output directory already exists. If set to true then imported data will be stored to already existing and possibly non empty directory. Type: BOOLEAN Sensitive: false Editable By: ANY Overrides: Connector with Name: kafka-connector Class: 1.99.7 Version: TO Supported Directions {4} link config 1: Name: linkConfig Label: Kafka cluster Help: Configuration options describing Kafka cluster. Input 1: Name: linkConfig.brokerList Label: Kafka brokers Help: Comma-separated list of Kafka brokers in the form of host:port. It doesn't need to contain all brokers, but at least two are recommended for high availability Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 1024 Input 2: Name: linkConfig.zookeeperConnect Label: Zookeeper quorum Help: Address of Zookeeper used by the Kafka cluster. Usually host:port. Multiple zookeeper nodes are supported. If Kafka is stored in its own znode use host:portkafka Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 TO Job config 1: Name: toJobConfig Label: Output configuration Help: Configuration necessary when writing data to Kafka. Input 1: Name: toJobConfig.topic Label: Topic Help: Name of Kafka topic where data will be written into. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Connector with Name: sftp-connector Class: 1.99.7 Version: TO Supported Directions {4} link config 1: Name: linkConfig Label: FTP Server configuration Help: Parameters required to connect to an SFTP server. Input 1: Name: linkConfig.server Label: Hostname Help: Hostname of the SFTP server. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 255 Input 2: Name: linkConfig.port Label: Port Help: Port for the SFTP server. Connector will use 22 if omitted. Type: INTEGER Sensitive: false Editable By: ANY Overrides: Input 3: Name: linkConfig.username Label: Username Help: Username that will be used to authenticate connection to SFTP serer. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 256 Input 4: Name: linkConfig.password Label: Password Help: Password that will be used to authenticate connection to the FTP Server. Type: STRING Sensitive: true Editable By: ANY Overrides: Size: 256 TO Job config 1: Name: toJobConfig Label: Output configuration Help: Parameters required to store data on the SFTP server. Input 1: Name: toJobConfig.outputDirectory Label: Output directory Help: Directory on the SFTP server to write data to. Type: STRING Sensitive: false Editable By: ANY Overrides: Size: 260 sqoop:000>
sqoop:000> create link -connector hdfs-connector 0 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Creating link for connector with name hdfs-connector Please fill following values to create new link object Name: hdfs HDFS cluster URI: hdfs://127.0.0.1:9000 Conf directory: Additional configs:: There are currently 0 values in the map: entry# New link was successfully created with validation status OK and name hdfs sqoop:000>
sqoop:000> show link +------+----------------+---------+ | Name | Connector Name | Enabled | +------+----------------+---------+ | hdfs | hdfs-connector | true | +------+----------------+---------+
sqoop:000> create link -connector generic-jdbc-connector Creating link for connector with name generic-jdbc-connector Please fill following values to create new link object Name: mysql Database connection Driver class: com.mysql.jdbc.Driver Connection String: jdbc:mysql://127.0.0.1:3306/test Username: test Password: **** Fetch Size: Connection Properties: There are currently 0 values in the map: entry# SQL Dialect Identifier enclose: New link was successfully created with validation status OK and name mysql
sqoop:000> show link +-------+------------------------+---------+ | Name | Connector Name | Enabled | +-------+------------------------+---------+ | mysql | generic-jdbc-connector | true | | hdfs | hdfs-connector | true | +-------+------------------------+---------+
sqoop:000> create job -f "mysql" -t "hdfs" Creating job for links with from name mysql and to name hdfs Please fill following values to create new job object Name: from-mysql-to-hdfs Database source Schema name: test Table name: member SQL statement: Column names: There are currently 0 values in the list: element# Partition column: Partition column nullable: Boundary query: Incremental read Check column: Last value: Target configuration Override null value: Null value: File format: 0 : TEXT_FILE 1 : SEQUENCE_FILE 2 : PARQUET_FILE Choose: 0 Compression codec: 0 : NONE 1 : DEFAULT 2 : DEFLATE 3 : GZIP 4 : BZIP2 5 : LZO 6 : LZ4 7 : SNAPPY 8 : CUSTOM Choose: 0 Custom codec: Output directory: /sqoop/member Append mode: Throttling resources Extractors: Loaders: Classpath configuration Extra mapper jars: There are currently 0 values in the list: element# New job was successfully created with validation status OK and name from-mysql-to-hdfs
sqoop:000> show job +----+--------------------+--------------------------------+-----------------------+---------+ | Id | Name | From Connector | To Connector | Enabled | +----+--------------------+--------------------------------+-----------------------+---------+ | 1 | from-mysql-to-hdfs | mysql (generic-jdbc-connector) | hdfs (hdfs-connector) | true | +----+--------------------+--------------------------------+-----------------------+---------+
sqoop:000> start job -n from-mysql-to-hdfs sqoop:000> start job -n from-mysql-to-hdfs Submission details Job Name: from-mysql-to-hdfs Server URL: http://localhost:12000/sqoop/ Created by: hadoop Creation date: 2017-07-22 23:18:02 CST Lastly updated by: hadoop External ID: job_1499236611045_0001 http://iZj6ciilv2rcpgauqg2uuwZ:8088/proxy/application_1499236611045_0001/ 2017-07-22 23:18:02 CST: BOOTING - Progress is not available
啟動後進入HDFS查看導入情況
[hadoop@netkiller ~]$ hdfs dfs -ls /sqoop [hadoop@netkiller ~]$ hdfs dfs -ls /member Found 10 items -rw-r--r-- 3 hadoop supergroup 0 2017-07-22 23:18 /member/310af608-5533-4bc2-bfb8-eaa45470b04d.txt -rw-r--r-- 3 hadoop supergroup 48 2017-07-22 23:18 /member/36bc39a5-bc73-4065-a361-ff2d61c4922c.txt -rw-r--r-- 3 hadoop supergroup 0 2017-07-22 23:18 /member/3e855400-84a9-422d-b50c-1baa9666a719.txt -rw-r--r-- 3 hadoop supergroup 140 2017-07-22 23:18 /member/3e8dad92-e0f1-4a74-a337-642cf4e6d634.txt -rw-r--r-- 3 hadoop supergroup 55 2017-07-22 23:18 /member/4a9f47f1-0413-4149-a93a-ed8b51efbc87.txt -rw-r--r-- 3 hadoop supergroup 0 2017-07-22 23:18 /member/4dc5bfe7-1cd9-4d9b-96a8-07e82ed79a71.txt -rw-r--r-- 3 hadoop supergroup 0 2017-07-22 23:18 /member/60dbcc60-61f2-4433-af39-1dfdfc048940.txt -rw-r--r-- 3 hadoop supergroup 0 2017-07-22 23:18 /member/6d02ed89-94d9-4d4b-87ed-d5da9d2bf9fe.txt -rw-r--r-- 3 hadoop supergroup 209 2017-07-22 23:18 /member/cf7b7185-3ab6-4077-943a-26228b769c57.txt -rw-r--r-- 3 hadoop supergroup 0 2017-07-22 23:18 /member/f2e0780d-ad33-4b35-a1c7-b3fbc23e303d.txt
sqoop:000> update link -n mysql Updating link with name mysql Please update link: Name: mysql Database connection Driver class: com.mysql.jdbc.Driver Connection String: jdbc:mysql://127.0.0.1:3306/test Username: test Password: **** Fetch Size: Connection Properties: There are currently 0 values in the map: entry# SQL Dialect Identifier enclose: link was successfully updated with status OK