8.2. Hadoop Common/HDFS Known Issues

  • BUG-30030: Incomplete installation of hadoop-httpfs.

    Problem: hadoop-httpfs RPM scripts are incomplete for HDP 2.2. This affects proxy setup that allows Hue to interact with HA clusters.

    Workaround: Have Falcon write to target clusters with wire encryption disabled.

    Workaround: The following steps should be executed on the node that will be hosting the HttpFS service. This assumes that HDP 2.2 has already been deployed and configured.

    1. Install the hadoop_2_2_0_0_2041-httpfs package:

      yum install hadoop-httpfs
    2. Create a symlink in /usr/lib to the location where httpfs is installed:

      ln -s /usr/hdp/current/hadoop-client/../hadoop-httpfs /usr/lib/hadoop-httpfs
    3. Create a symlink to libexec:

      ln -s /usr/hdp/current/hadoop-client/../hadoop/libexec /usr/lib/hadoop-httpfs/libexec
    4. Create configuration directories and copy the default set of configurations:

      mkdir -p /etc/hadoop-httpfs/conf
      mkdir -p /etc/hadoop-httpfs/tomcat-deployment/conf
      cp -r /usr/hdp/2.2.0.0-2041/etc/hadoop-httpfs/conf.empty/* /etc/hadoop-httpfs/conf/
      cp -r /usr/hdp/2.2.0.0-2041/etc/hadoop-httpfs/tomcat-deployment.dist/conf/* /etc/hadoop-httpfs/tomcat-deployment/conf
    5. Enable the init.d script for HttpFS by creating the following symlink:

      ln -s /usr/hdp/2.2.0.0-2041/etc/rc.d/init.d/hadoop-httpfs /etc/init.d/hadoop-httpfs
    6. Create the following symlinks to point to the Tomcat libraries and binary scripts that HttpFS will use:

      ln -s /usr/lib/bigtop-tomcat/lib /etc/hadoop-httpfs/tomcat-deployment/libln -s /usr/lib/bigtop-tomcat/bin /etc/hadoop-httpfs/tomcat-deployment/bin
    7. Start the HttpFs server:

      service hadoop-httpfs start