install python hdfs API library - libpyhdfs

** install libpyhdfs $ sudo apt-get install libhdfs4-dev

$ svn co http://libpyhdfs.googlecode.com/svn/branches/bundle/ libpyhdfs

$ cd libpyhdfs/lib

$ wget http://libpyhdfs.googlecode.com/files/commons-logging-1.0.4.jar

$ wget http://libpyhdfs.googlecode.com/files/hadoop-0.20.1-core.jar

$ wget http://libpyhdfs.googlecode.com/files/libhdfs.so.0 ==> you need to copy your own installed hadoop lib. In my case, I copyed /usr/local/hadoop-1.0.4/c++/Linux-amd64-64/lib/libhdfs.so.0 to libpyhdfs/lib

$ ln -s libhdfs.so.0 libhdfs.so

$ cd ..

# python setup.py install --prefix="/usr/local"

If you see the following error:

/usr/lib/jvm/java-6-sun/include/jni.h:27:20: error: jni_md.h: No such file or directory

Edit /usr/lib/jvm/java-6-sun/include/jni.h

change - 27 #include "jni_md.h"

into + 27 #include "linux/jni_md.h"

** Run test script

$ cd test

$ python pyhdfs_test.py

If you see the following error:

ImportError:libhdfs.so.0:cannot open shared object file:No such file or directory
you need to edit/etc/ld.so.conf.

Add  mentioned above /usr/local/hadoop-1.0.4/c++/Linux-amd64-64/lib/ directory to it.

In my case, it looks like ...

$cat /etc/ld.so.conf

/usr/local/hadoop-1.0.4/c++/Linux-amd64-64/lib/
include /etc/ld.so.conf.d/*.conf

Now you meed to run below command.

$ sudo/sbin/ldconfig -v

If you see error as below...
Environment variable CLASSPATH not set!
Just add CLASSPATH to /etc/profile as below...
export CLASSPATH=$(find ${HADOOP_HOME} -name *.jar | tr '\n' ':')

Now it should work.



Reference Site https://code.google.com/p/libpyhdfs/source/browse/branches/bundle/INSTALL
http://www.opendebug.com/article/483718