4.2. Install HDP

Use the following instructions to install HDP on your cluster hardware. Ensure that you specify the virtual machines(configured in the previous section) as your NameNode and as your JobTracker.

  1. Download Hortonworks Data Platform (HDP) using the instructions provided here.

  2. Use the fully qualified domain name (FQDN) of the virtual machine for configuring the host names (see: Installing, Configuring, and Deploying the Cluster - Assign Masters.).

    [Note]Note

    HDP might not identify the NameNode or JobTracker VMs automatically and it is therefore important to note down FQDN (IP address and DNS name) of these VMs.

  3. Specify shared storage for the NameNode’s directories (see: HDP (Ambari) - Customize Services).

  4. Do not use the NameNode VM for running any other master daemon.

  5. Do not use the JobTracker VM for running any other master daemon.

  6. While deploying Ambari, add the following custom configuration parameters for Full-Stack HA for both the NameNode and the JobTracker.

    1. Add the following properties and values to custom hdfs-site section on the dashboard:

      Table 3.1. Custom properties for hdfs-site section
      Key Value Notes
      dfs.client.retry.policy.enabled true Enable the HDFS client retry policy.
      dfs.namenode.edits.toleration.length 8192 Configure protection for NameNode edit log.
      dfs.safemode.extension 10 Configure safe mode extension time.
      dfs.persist.blocks true Ensure that the allocated DFS blocks persist across multiple fail overs.
      dfs.blockreport.initialDelay 10 Configure delay for first block report.

    2. Add the following properties and values to custom mapred-site section on the dashboard:

      Table 3.2. Custom properties for mapred-site section
      Key Value Notes
      mapreduce.jt.hdfs.monitor.enable true Enable the JobTracker’s safe mode functionality.
      mapreduce.jobclient.retry.policy.enabled true Enable retry for JobTracker clients (when the JobTracker is in safe mode).
      mapred.jobtracker.restart.recover true Enable recovery of JobTracker’s queue after it is restarted.

    3. Add the following properties and values to custom core-site section on the dashboard:

      Table 3.3. Custom properties for core-site section
      Key Value Notes
      fs.checkpoint.period 3600 Configure checkpoint interval so that the checkpoint is performed on an hourly basis.


loading table of contents...