5.5. Run Hadoop Smoke Tests

To smoke test your Hadoop upgrade, you can run the following MapReduce job.

Run this command as regular user. The job uses MapReduce to write 100MB of data into HDFS with RandomWriter. hadoop jar /usr/lib/hadoop-mapreduce/*examples*.jar randomwriter -Dtest.randomwrite.total_bytes=100000000 test-after-upgrade You should see messages similar to:

map 0% reduce 0%
…
map 100% reduce 100%
Job …. completed successfully

You just submitted your first MapReduce job in HDP 2.x. Good job! The next steps are to upgrade your other components.

Basic troubleshooting:

  1. To find the number of active nodes and NodeManagers, access the ResourceManager web UI:

    http://<resource manager host>:8088/cluster/nodes

    The number of active nodes should be equal to the number of nodemanagers.

  2. Error messages. Access the ApplicationMaster WebUI to view the container logs.

    1. Looking at your console logs for MapReduce job, there is a line in this format:

       13/10/02 17:57:21 INFO mapreduce.Job: The url to track the job: http://<resource manager host>:8088/proxy/application_1380673658357_0007/

    2. Go to the URL and select the job link.

    3. Select the logs link under ApplicationMaster table. It will redirect you to the container logs. Error messages display here.


loading table of contents...