Monday, June 9, 2014

More Little Lessons in Hadoop

Previously, I had posted on some common issues encountered when installing and running Hadoop for the first time.
Now, in the joyful experience of attempting a multi-node setup on a simple virtual cluster, many more errors, unexpected consequences, and curios of server-related struggle have been forthcoming.
I hope that in relaying these attempts, failures, and (admittedly sparse) triumphs through this medium, I can allay some of the angst experienced by others in the future.

Good luck, all Hadooper-troopers.

Can't yum/apt-get? 

Depending on OS version and pre-existing settings (especially when working on a stripped virtual image), your installer service may not be able to find certain packages.


For a general inability to install, try the configurations here for setting up the nginx repo. 

A particular problem I ran across was installing the repository management on CentOS, either "python-software-properties" or "software-properties-common" depending on version.
Turns out, you don't need it, contrary to the results I found on several google searches. 

Specifically, if you're following Michael Noll's tutorial, and you can't install the software properties, try doing a yum search for Java, and you'll probably get results.

However, if for some reason you don't, here are some links that may be of use:

Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

This may be due to an incorrect permissions setting, which you can solve via these directions. However, especially if you've just sourced eucarc to get going, you may want to make sure you're in your home directory...yes, I did make that mistake; it's quite frustrating until it suddenly becomes embarrassing.

For more general help on SSH through Ubuntu:

/usr/lib/jvm/java.... Not a directory

Most likely a mistyped or mixed-up Java directory, especially if you've switched Java versions recently or are setting up a new user.
Check your hadoop-env.sh file, and modify the path as necessary (probably ending with the jre directory).

java.net.UnknownHostException on hadoop namenode -format


No comments:

Post a Comment