Reference
Also available as:
PDF

Option I - Mirror server has no access to the Internet

Complete the following instructions to set up a mirror server that has no access to the Internet:

  1. Check Your Prerequisites.

    Select a mirror server host with the following characteristics:

    • The server OS is Debian 7, CentOS (6,7), RHEL (6,7), Oracle Linux(6,7), SLES (11,12), or Ubuntu (12,14,16), and has several GB of storage available.

    • This server and the cluster nodes are all running the same OS.

      [Note]Note

      To support repository mirroring for heterogeneous clusters requires a more complex procedure than the one documented here.

    • The firewall lets all cluster nodes (the servers on which you want to install HDP) access this serve

  2. Install the Repos.

    1. Use a workstation with access to the Internet and download the tarball image of the appropriate Hortonworks yum repository.

      Table 3.2. Hortonworks Yum Repositories

      Cluster OSHDP Repository Tarballs
      Debian 7

      wget http://public-repo-1.hortonworks.com/HDP/debian7/2.x/updates/2.6.5.0/HDP-2.6.5.0-debian7-deb.tar.gz

      wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/debian7/HDP-UTILS-1.1.0.21-debian7.tar.gz

      RHEL/CentOS/Oracle LINUX 6

      wget http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos6-rpm.tar.gz

      wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6/HDP-UTILS-1.1.0.21-centos6.tar.gz

      RHEL/CentOS/Oracle LINUX 7

      wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz

      wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz

      SLES 11 SP3/SP4

      wget http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.6.5.0/HDP-2.6.5.0-suse11sp3-rpm.tar.gz

      wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/suse11sp3/HDP-UTILS-1.1.0.21-suse11sp3.tar.gz

      SLES 12

      wget http://public-repo-1.hortonworks.com/HDP/sles12/2.x/updates/2.6.5.0/HDP-2.6.5.0-sles12-rpm.tar.gz

      wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/sles12/HDP-UTILS-1.1.0.21-sles12.tar.gz

      Ubuntu 12

      wget http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/updates/2.6.5.0/HDP-2.6.5.0-ubuntu12-deb.tar.gz

      wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu12/HDP-UTILS-1.1.0.21-ubuntu12.tar.gz

      Ubuntu 14

      wget http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.6.5.0/HDP-2.6.5.0-ubuntu14-deb.tar.gz

      wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14/HDP-UTILS-1.1.0.21-ubuntu14.tar.gz

      Ubuntu 16

      wget http://public-repo-1.hortonworks.com/HDP/ubuntu16/2.x/updates/2.6.5.0/HDP-2.6.5.0-ubuntu16-deb.tar.gz

      wget http://public-repo-1.hortonworks.com.s3.amazonaws.com/HDP-UTILS-1.1.0.21/repos/ubuntu16/HDP-UTILS-1.1.0.21-ubuntu16.tar.gz


    2. Create an HTTP server.

      • On the mirror server, install an HTTP server (such as Apache httpd) using the instructions provided here.

      • Activate this web server.

      • Ensure that the firewall settings (if any) allow inbound HTTP access from your cluster nodes to your mirror server.

        [Note]Note

        If you are using EC2, make sure that SELinux is disabled.

        If you are using EC2, make sure that SELinux is disabled.

    3. On your mirror server, create a directory for your web server.

      • For example, from a shell window, type:

        • ForRHEL/CentOS/Oracle:

          mkdir –p /var/www/html/hdp/
        • For SLES:

          mkdir –p /srv/www/htdocs/rpms
        • For Ubuntu:

          mkdir –p /var/www/html/hdp/
      • If you are using a symlink, enable thefollowsymlinks on your web server.

    4. Copy the HDP Repository Tarball to the directory created in step 3, and untar it.

    5. Verify the configuration.

      • The configuration is successful, if you can access the above directory through your web browser.

        To test this out, browse to the following location: http://$yourwebserver/hdp/$os/HDP-2.6.5.0/.

        You should see directory listing for all the HDP components along with the RPMs at: $os/HDP-2.6.5.0.

        [Note]Note

        If you are installing a 2.x.0 release, use: http://$yourwebserver /hdp/ $os /2.x/GA

        If you are installing a 2.x.x release, use: http://$yourwebserver /hdp/ $os /2.x/updates

        where

        • $os can be debian7, centos6, centos7, suse11sp3, sles12, ubuntu12, ubuntu14, or ubuntu16. Use the following options table for $osparameter:

          Table 3.3. HDP Component Options

          Operating SystemValue
          Debian 7debian7
          RHEL 6centos6
          RHEL 7centos7
          SLES 11 SP3/SP4suse11sp3
          SLES 12sles12
          Ubuntu 12ubuntu12
          Ubuntu 14ubuntu14
          Ubuntu 16ubuntu16


    6. Configure the yum clients on all the nodes in your cluster.

      • Fetch the yum configuration file from your mirror server.

        http://$yourwebserver /hdp/$os/2.x/updates/2.6.5.0/hdp.repo
      • Store the hdp.repo file to a temporary location.

      • Edit the hdp.repo file changing the value of the base url property to point to your local repositories based on your cluster OS.

        where

        • $yourwebserver is the FQDN of your local mirror server.

        • $os can be centos6, centos7, suse11sp3, sles12, ubuntu12, ubuntu14, or ubuntu16. Use the following options table for $os parameter:

          Table 3.4. Yum Client Options

          Operating SystemValue
          Debian 7debian7
          RHEL 6centos6
          RHEL 7centos7
          SLES 11 SP3/SP4suse11sp3
          SLES 12sles12
          Ubuntu 12ubuntu12
          Ubuntu 14ubuntu14
          Ubuntu 16ubuntu16


        • Use scp or pdsh to copy the client yum configuration file to /etc/yum.repos.d/ directory on every node in the cluster.

      • [Conditional]: If you have multiple repositories configured in your environment, deploy the following plugin on all the nodes in your cluster.

        • Install the plugin.

          • For RHEL and CentOs

            yum install yum-plugin-priorities
        • Edit the /etc/yum/pluginconf.d/priorities.conf file to add the following:

          [main]
          enabled=1
          gpgcheck=0