2.4.5.  Upgrade YARN and MapReduce2

  1. Prepare MR2 and YARN for work. Execute HDFS commands on any host.

    • Create mapreduce dir in hdfs.

      su -l <HDFS_USER> -c "hdfs dfs -mkdir -p /hdp/apps/2.3.x.y-z/mapreduce/"

    • Copy new mapreduce.tar.gz to HDFS mapreduce dir.

      su -l <HDFS_USER> -c "hdfs dfs -put /usr/hdp/current/hadoop-client/mapreduce.tar.gz /hdp/apps/2.3.x.y-z/mapreduce/."

    • Copy new hadoop-streaming.jar to HDFS mapreduce dir.

      su -l <HDFS_USER> -c "hdfs dfs -put /usr/hdp/current/hadoop-mapreduce-client/hadoop-streaming.jar /hdp/apps/2.3.x.y-z/mapreduce/."

    • Grant permissions for created mapreduce dir in hdfs.

      su -l <HDFS_USER> -c "hdfs dfs -chown -R <HDFS_USER>:<HADOOP_GROUP> /hdp"
      su -l <HDFS_USER> -c "hdfs dfs -chmod -R 555 /hdp/apps/2.3.x.y-z/mapreduce"
      su -l <HDFS_USER> -c "hdfs dfs -chmod -R 444 /hdp/apps/2.3.x.y-z/mapreduce/mapreduce.tar.gz"
      su -l <HDFS_USER> -c "hdfs dfs -chmod -R 444 /hdp/apps/2.3.x.y-z/mapreduce/hadoop-streaming.jar"
    • Update YARN Configuration Properties for HDP 2.3

      From Ambari Web, browse to > Services > YARN > Configs. Filter for the following properties, and add these properties if they do not already exist:

      Name

      Value

      hadoop.registry.zk.quorum

      The comma-separated list of ZooKeeper hosts defining the ZooKeeper quorum.

      For example:

      ZooKeeperHost1:2181, ZooKeeperHost2:2181

      yarn.resourcemanager.zk-address

      The comma-separated list of ZooKeeper hosts defining the ZooKeeper quorum.

      For example:

      ZooKeeperHost1:2181, ZooKeeperHost2:2181

  2. Manually clear the ResourceManager state store.

    su -l <YARN_USER> -c "yarn resourcemanager -format-state-store"

    where <YARN_USER> is the YARN Service user. For example, yarn.

  3. Using Ambari Web, browse to Services > YARN and select Start from the Service Actions menu. Repeat for MapReduce2.

  4. Run the smoke checks. Using Ambari Web, browse to Services > YARN and select Run Service Check from the Service Actions menu. Confirm the check passes. Repeat for MapReduce2.


loading table of contents...