2. Upgrade Hadoop

  1. On all nodes, clean the yum repository.

    • For RHEL/CentOS:

      yum clean all 
    • For SLES:

      zypper clean --all
    • For Ubuntu:

      apt-get clean all

  2. Uninstall the HDP 1.x packages.

    • For RHEL/CentOS:

      yum erase hadoop-pipes hadoop-sbin hadoop-native oozie 
    • For SLES:

      zypper rm hadoop-pipes hadoop-sbin hadoop-native oozie hbase hadoop*
    • For Ubuntu:

      apt-get remove hadoop-pipes hadoop-sbin hadoop-native oozie hbase hadoop* 

  3. Configure your repository.

    The standard HDP install fetches the software from a remote yum repository
 over the Internet. To use this option, you must set up access to the remote
 repository and have an available Internet connection for each of your
 hosts

    [Note]Note

    If your cluster does not have access to the Internet, or you are
 creating a large cluster and you want to conserve bandwidth, you can
 instead provide a local copy of the HDP repository that your hosts can
 access. For more information, see Deployment Strategies for Data Centers with
 Firewalls, a separate document in this set.

    1. For each node in your cluster, download the yum repo configuration
 file hdp.repo. From a terminal window,
 enter the following wget command.

      • For RHEL/CentOS/Oracle Linux 5
:

        
wget http://public-repo-1.hortonworks.com/HDP/centos5/2.x/2.1-latest/hdp.repo -O /etc/yum.repos.d/hdp.repo


      • For RHEL/CentOS/Oracle Linux 6:

        
wget http://public-repo-1.hortonworks.com/HDP/centos6/2.x/2.1-latest/hdp.repo -O /etc/yum.repos.d/hdp.repo


      • For SLES 11:

        
wget http://public-repo-1.hortonworks.com/HDP/suse11/2.x/2.1-latest/hdp.repo -O /etc/zypp/repos.d/hdp.repo


      • For Ubuntu:

        
wget http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/hdp.list -O /etc/apt-get/repos.d/hdp.list


    2. Confirm the HDP repository is configured by checking the repo
 list.

      • For RHEL/CentOS/Oracle Linux:


        yum repolist
      • For SLES:


        zypper repos

      • For Ubuntu:


        apt-get list

  4. Use the HDP Utility script to calculate memory configuration settings.

  5. Install Hadoop

    • For RHEL/CentOS/Oracle Linux:

      yum upgrade hadoop*

    • For SLES:

      zypper install hadoop* hadoop-hdfs hadoop-lzo
    • For Ubuntu:

      apt-get update
      apt-get install hadoop hadoop-hdfs libhdfs0 libhdfs0-dev hadoop-yarn hadoop-mapreduce hadoop-client opensslliblzo2-2 liblzo2-dev hadoop-lzo 
  6. Install YARN

    • For RHEL/CentOS/Oracle Linux:

      yum install hadoop-mapreduce hadoop-yarn

    • For SLES:

      zypper install hadoop-mapreduce hadoop-yarn
    • For Ubuntu:

      apt-get install hadoop-mapreduce hadoop-yarn
  7. Verify HDP 2.x packages have installed successfully.

    • For RHEL/CentOS/Oracle Linux:

      yum list hadoop*|grep HDP-2 

    • For SLES:

      zypper pa|grep HDP-2 

      Verify that you have HDP 2.x installed:

      hadoop version

      You may need to add /etc/hadoop/conf/hadoop-env.sh in /usr/bin/hadoop for $JAVA_HOME.

    • For Ubuntu:

       dpkg -s HDP-2 | grep Status 


loading table of contents...