5. Smoke Test MapReduce

  1. Try browsing to the ResourceManager:

    http://$resourcemanager.full.hostname:8088/
  2. Create a $CLIENT_USER in all the nodes and add it in users group.

    useradd client
    usermod -a -G users client
  3. As the HDFS user, create a /user/$CLIENT_USER.

    sudo su $HDFS_USER 
    hdfs dfs -mkdir /user/$CLIENT_USER 
    hdfs dfs -chown $CLIENT_USER:$CLIENT_USER /user/$CLIENT_USER 
    hdfs dfs -chmod -R 755 /user/$CLIENT_USER
  4. Run the smoke test as the $CLIENT_USER, using Terasort and sort 10GB of data.

    su $CLIENT_USER 
    /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-*.jar teragen 10000 tmp/teragenout 
    /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-*.jar terasort tmp/teragenout tmp/terasortout
    

loading table of contents...