Use the following instructions to configure Hadoop:
These configuration variables are under the [hadoop]
section in the
hue.ini
configuration file.
Configure HDFS Cluster.
Hue supports only one HDFS cluster currently.
Ensure that you define the HDFS cluster under the
[[[default]]]
sub-section. Use the following variables to configure the HDFS cluster:- fs_defaultfs
This is equivalent to
fs.defaultFS
(fs.default.name
) in Hadoop configuration.- webhdfs_url
You can also set this to be the HttpFS URL. The default value is the HTTP port on the NameNode.
- hadoop_hdfs_home
This is the home of your Hadoop HDFS installation. It is the root of the Hadoop untarred directory or usually
/usr/lib/hadoop
.- hadoop_bin
Use this as the HDFS Hadoop launcher script, which is usually
/usr/bin/hadoop
.- hadoop_conf_dir
This is the configuration directory of the HDFS, typically
/etc/hadoop/conf
.
Configure YARN (MR2) Cluster.
Hue supports only one YARN cluster currently.
Ensure that you define the YARN cluster under the
[[[default]]]
sub-section. Use the following variables to configure the YARN cluster:- resourcemanager_host
The host running the ResourceManager.
- resourcemanager_port
The port for the ResourceManager IPC service.
- submit_to
Set this property to true. Hue will be submitting jobs to this Yarn cluster. But note that JobBrowser will not be able to show MR2 jobs.
- hadoop_mapred_home
This is the home of your Hadoop MapReduce installation. It is the root of HDP Hadoop-MapReduce directory (
/usr/lib/hadoop-mapreduce
). Ifsubmit_to
istrue
for this cluster, this configuration value is set as the$HADOOP_MAPRED_HOME
for BeeswaxServer and child shell processes.- hadoop_bin
Use this as the Yarn/MR2 Hadoop launcher script (
/usr/bin/hadoop
).- hadoop_conf_dir
This is the configuration directory of the Yarn/MR2 service, typically set to
/etc/hadoop/conf
.