whoami

hadoop file read write 시 인코딩을 변경하는 방법             mapred.child.java.opts     -Xmx200m -Dfile.encoding=utf-8     Java opts for the task tracker child processes.  Subsumes     ‘mapred.child.heap.size’ (If a mapred.child.heap.size value is found     in a configuration, its maximum heap size will be used and a warning     emitted that heap.size has been deprecated). Also, the following symbols,     if present, will be interpolated: @taskid@ is replaced by current TaskID;     and @port@ will be replaced by mapred.task.tracker.report.port + 1 (A second     child will fail with a port-in-use if mapred.tasktracker.tasks.maximum is     greater than one). Any other occurrences of ‘@’ will go unchanged. For     example, to enable verbose gc logging to a file named for the taskid in     /tmp and to set the heap maximum to be a gigabyte, pass a ‘...

sqoop 을 이용하여 RDBMS 로 export 시 duplicate error 가 발생할 경우가 있다. --update-key {key}  --update-mode allowinsert   이 옵션을 사용하면 {key}에 해당하는 레코드가 존재하면 update 를 진행하고 존재하지 않으면 insert 를 진행한다. --update-mode 를 updateonly 로 설정하면 insert 는 진행하지 않는다. sqoop export \ --connect jdbc:mysql://airawat-mysqlserver-node/employees \ --username  myUID  \ --password  myPWD  \ --table employees_export  \ --update-key emp_no \ --update-mode allowinsert \ --export-dir /user/airawat/sqoop-mysql/Employees 자세한것은 http://hadooped.blogspot.kr/2013/06/apache-sqoop-part-3-data-transfer.html 여기 참조