3. Manually Creating a Cluster Properties File

Use the following instructions to manually configure the cluster properties file for deploying HDP from the command-line or in a script:

Create a clusterproperties.txt file or use the sample clusterproperties.txt file extracted from the HDP Installation zip file.

Add the properties to the clusterproperties.txt file as described in the table below:

Important

	Important
All properties in the `clusterproperties.txt` file must be separated by a newline character. Directory paths cannot contain whitespace characters. For example, `C:\Program Files\Hadoop` is an invalid directory path for HDP. Use Fully Qualified Domain Names (FQDN) for specifying the network host name for each cluster host. The FQDN is a DNS name that uniquely identifies the computer on the network. By default, it is a concatenation of the host name, the primary DNS suffix, and a period. When specifying the host lists in the `clusterproperties.txt` file, if the hosts are multi-homed or have multiple NIC cards, make sure that each name or IP address by which you specify the hosts is the preferred name or IP address by which the hosts can communicate among themselves. In other words, these should be the addresses used internal to the cluster, not those used for addressing cluster nodes from outside the cluster. To Enable NameNode HA, you must include the HA properties and exclude the SECONDARY_NAMENODE_HOST definition.

All properties in the clusterproperties.txt file must be separated by a newline character.
Directory paths cannot contain whitespace characters.
For example, C:\Program Files\Hadoop is an invalid directory path for HDP.
Use Fully Qualified Domain Names (FQDN) for specifying the network host name for each cluster host. The FQDN is a DNS name that uniquely identifies the computer on the network. By default, it is a concatenation of the host name, the primary DNS suffix, and a period.
When specifying the host lists in the clusterproperties.txt file, if the hosts are multi-homed or have multiple NIC cards, make sure that each name or IP address by which you specify the hosts is the preferred name or IP address by which the hosts can communicate among themselves. In other words, these should be the addresses used internal to the cluster, not those used for addressing cluster nodes from outside the cluster.
To Enable NameNode HA, you must include the HA properties and exclude the SECONDARY_NAMENODE_HOST definition.

Configuration Values for MSI Installer

Configuration Property Name	Description	Example value	Mandatory/ Optional/ Conditional
HDP_LOG_DIR	HDP's operational logs are written to this directory on each cluster host. Ensure that you have sufficient disk space for storing these log files.	`d:\hadoop\logs`	Mandatory
HDP_DATA_DIR	HDP data will be stored in this directory on each cluster node. You can add multiple comma-separated data locations for multiple data directories.	`d:\hdp\data`	Mandatory
NAMENODE_HOST	The FQDN for the cluster node that will run the NameNode master service.	`NAMENODE-MASTER.acme.com`	Mandatory
SECONDARY_NAMENODE_HOST	The FQDN for the cluster node that will run the Secondary NameNode master service.	`SECONDARY-NN-MASTER.acme.com`	Mandatory when no HA
RESOURCEMANAGER_HOST	The FQDN for the cluster node that will run the YARN Resource Manager master service.	`RESOURCE-MANAGER.acme.com`	Mandatory
HIVE_SERVER_HOST	The FQDN for the cluster node that will run the Hive Server master service.	`HIVE-SERVER-MASTER.acme.com`	Mandatory
OOZIE_SERVER_HOST	The FQDN for the cluster node that will run the Oozie Server master service.	`OOZIE-SERVER-MASTER.acme.com`	Mandatory
WEBHCAT_HOST	The FQDN for the cluster node that will run the WebHCat master service.	`WEBHCAT-MASTER.acme.com`	Mandatory
FLUME_HOSTS	A comma-separated list of FQDN for those cluster nodes that will run the Flume service.	`FLUME-SERVICE1.acme.com, FLUME-SERVICE2.acme.com, FLUME-SERVICE3.acme.com`	Mandatory
HBASE_MASTER	The FQDN for the cluster node that will run the HBase master.	`HBASE-MASTER.acme.com`	Mandatory
HBASE_REGIONSERVERS	A comma-separated list of FQDN for those cluster nodes that will run the HBase Region Server services.	`slave1.acme.com, slave2.acme.com, slave3.acme.com`	Mandatory
SLAVE_HOSTS	A comma-separated list of FQDN for those cluster nodes that will run the DataNode and TaskTracker services.	`slave1.acme.com, slave2.acme.com, slave3.acme.com`	Mandatory
ZOOKEEPER_HOSTS	A comma-separated list of FQDN for those cluster nodes that will run the ZooKeeper hosts.	`ZOOKEEEPER-HOST.acme.com`	Mandatory
FALCON_HOSTS	A comma-separated list of FQDN for those cluster nodes that will run the Falcon hosts.	`falcon.acme.com, falcon1.acme.com, falcon2.acme.com`	Optional
KNOX_HOST	The FQDN of the Knox Gateway host.	`KNOX-HOST.acme.com`	Optional
IS_TEZ	Install the Tez component on Hive host.	`YES` or `NO`	Optional
IS_PHOENIX	Installs Phoenix on the HBase hosts.	`YES` or `NO`	Optional
ENABLE_LZO	Enables the LZO codec for compression in HBase cells.	`YES` or `NO`	Optional
DB_FLAVOR	Database type for Hive and Oozie metastores (allowed databases are SQL Server and Derby). To use default embedded Derby instance, set the value of this property to `derby`. To use an existing SQL Server instance as the metastore DB, set the value as `mssql`.	`mssql` or `derby`	Mandatory
DB_HOSTNAME	FQDN for the node where the metastore database service is installed. If using SQL Server, set the value to your SQL Server hostname. If using Derby for Hive metastore, set the value to HIVE_SERVER_HOST.	`sqlserver1.acme.com`	Mandatory
DB_PORT	This is an optional property required only if you are using SQL Server for Hive and Oozie metastores. By default, the database port is set to `1433`.	1433	Optional
HIVE_DB_NAME	Database for Hive metastore. If using SQL Server, ensure that you create the database on the SQL Server instance.	`hivedb`	Mandatory
HIVE_DB_USERNAME	User account credentials for Hive metastore database instance. Ensure that this user account has appropriate permissions.	`hive_user`	Mandatory
HIVE_DB_PASSWORD		`hive_pass`	Mandatory
OOZIE_DB_NAME	Database for Oozie metastore. If using SQL Server, ensure that you create the database on the SQL Server instance.	`ooziedb`	Mandatory
OOZIE_DB_USERNAME	User account credentials for Oozie metastore database instance. Ensure that this user account has appropriate permissions.	`oozie_user`	Mandatory
OOZIE_DB_PASSWORD		`oozie_pass`	Mandatory

The following snapshot illustrates a sample clusterproperties.txt file:

#Log directory
HDP_LOG_DIR=c:\hadoop\logs

#Data directory
HDP_DATA_DIR=c:\hdpdata

#hosts
NAMENODE_HOST=nn-host.acme.com
SECONDARY_NAMENODE_HOST=sec-nn-host.acme.com
RESOURCEMANAGER_HOST=resourcemgr-host.acme.com
HIVE_SERVER_HOST=hive-host.acme.com
OOZIE_SERVER_HOST=oozie-host.acme.com
WEBHCAT_HOST=webhcat-host.acme.com
SLAVE_HOSTS=slave-host.acme.com,slave1-host.acme.com, slave2-host.acme.com
ZOOKEEPER_HOSTS=zookeeper-host.acme.com
CLIENT_HOSTS=client-host.acme.com,client2-host.acme.com
IS_TEZ=yes
ENABLE_LZO=yes
HBASE_MASTER=hbase-host.acme.com
HBASE_REGIONSERVERS=hbase-host.acme.com,hbase2-host.acme.com
FLUME_HOSTS=flume-host.acme.com
FALCON_HOST=falcon-host.acme.com
KNOX_HOST=knox-host.acme.com
STORM_NIMBUS=storm-host.acme.com
STORM_SUPERVISORS=stormsup-host.acme.com
IS_PHOENIX=yes

#Database host
DB_FLAVOR=DERBY
DB_HOSTNAME=hive-host.acme.com
DB_PORT=1527

#Hive properties
HIVE_DB_NAME=hive
HIVE_DB_USERNAME=hive
HIVE_DB_PASSWORD=hive3

#Oozie properties
OOZIE_DB_NAME=oozie
OOZIE_DB_USERNAME=oozie
OOZIE_DB_PASSWORD=oozie

Legal notices