4. Create Local Directories

You must create local directories for YARN on each NodeManager host in your cluster (in HDP-2, the NodeManager replaces the TaskTracker) and set the appropriate permissions for the YARN log directories. If these directories do not exist, you can create them using the instructions on this page.

  1. Set the permissions in the yarn.nodemanager.local-dirs directories. Run these commands on all DataNodes in your cluster.

    chown -R yarn:hadoop ${yarn.nodemanager.local-dirs}
    chmod 755 ${yarn.nodemanager.local-dirs}

    where ${yarn.nodemanager.local-dirs} is your local directory.

  2. Change the permissions of the directories in yarn.nodemanager.log-dirs. If these directories do not exist, you can create them using the instructions on this page. Run these commands on all DataNodes in your cluster.

    chown -R yarn:hadoop ${yarn.nodemanager.log-dirs}
    chmod 755 ${yarn.nodemanager.log-dirs}

    where ${yarn.nodemanager.log-dirs} is your log directory.

  3. Create directories for YARN_LOG_DIR and YARN_PID_DIR.

    1. Open /etc/hadoop/conf/yarn-env.sh

    2. Write down your values for YARN_LOG_DIR and YARN_PID_DIR as the following instructions require values for the ${YARN_LOG_DIR} and ${YARN_PID_DIR}. For example in yarn-env.sh:

      YARN_LOG_DIR=/grid/0/var/log/hadoop/yarn
      YARN_PID_DIR=/grid/0/var/run/hadoop/yarn

  4. Make directories for ${YARN_LOG_DIR} and ${YARN_PID_DIR} and set the appropriate permissions for them.

    mkdir ${YARN_LOG_DIR}
    chown yarn:hadoop ${YARN_LOG_DIR}
    chmod 755 ${YARN_LOG_DIR}
    
    mkdir ${YARN_PID_DIR}
    chown yarn:hadoop ${YARN_PID_DIR}
    chmod 755 ${YARN_PID_DIR}

loading table of contents...