1.sqoop命令的执行方式:
(1). Python : retCode = subprocess.call(sqoopCmd, shell=True)
eg:
sqoopCmd = "sqoop import " \+ "--connect '" + dbConnStr + "' " \+ "--username '" + dbUser + "' " \+ "--password '" + dbPass + "' " \+ "--query \"" + sqoopQueryStr + "\" " \+ "--null-string '\\\\N' " \+ "--null-non-string '\\\\N' " \+ "--target-dir '" + sqoopTargetFile + "' " \+ "--hive-drop-import-delims " \+ "--fields-terminated-by '" + sqoopFieldsSep + "' " \+ "--lines-terminated-by '" + sqoopLinesSep + "' " \+ "--append " \+ "--temporary-rootdir '" + sqoopTmpDir + "' " \+ "--m " + sqoopMaps \logging.info("sqoop cmd is : %s" % (sqoopCmd))retCode = subprocess.call(sqoopCmd, shell=True)
(2). Shell : 编写shell脚本,直接将参数写入shell中
eg:
#!/bin/bash#CURR_DATE=`date +"%Y-%m-%d %H:%M:%S"`------>不能使用
v_sql="insert into origin_ennenergy_energytrade.test2 values('"$(date +"%Y-%m-%d %H:%M:%S")"','"Y"')"
echo $v_sql#insert into origin_ennenergy_energytrade.test2 values('2016-08-09 10:39:44','Y')hive -e "$v_sql;"
sqoop export --connect jdbc:mysql://ip:3306/test23?characterEncoding=utf8 --username root --password 123--table test2--export-dir /user/hive/warehouse/origin_ennenergy_energytrade.db/test2/* --input-fields-terminated-by "\t" --update-mode allowinsert --update-key times;
执行shell
sh test.sh
2.Sqoop参数详见:
http://sqoop.apache.org/docs/1.4.5/SqoopUserGuide.html