1.  HDP MSI Installer Properties

This section explains the HDP MSI installer command line options and HDP public properties to use when installing a multi-node Hadoop Cluster.

The format of the HDP MSI Installer command is:

msiexec /qn /lv "log_file" /i "msi_file" MSIUSEREALADMINDETECTION=1 HDP_DIR="install_dir" HDP_LAYOUT="cluster_properties_file" HDP_USER_PASSWORD="password" DESTROY_DATA="YES_OR_NO" HDP="YES_OR_NO" FLUME="YES_or_NO" HBASE="YES_or_NO" KNOX="YES_or_NO" KNOX_MASTER_SECRET="secret" FALCON="YES_or_NO" STORM="YES_or_NO"

where:

  • msiexec /qn /lv "log_file" /i "msi_file MSIUSEREALADMINDETECTION=1" is the standard installer options recommended by Hortonworks:

    • /qn (quiet, no UI) suppresses the HDP Setup Window. Use /qb (quiet basic) to suppress the HDP Setup and show a progress bar.

    • /lv "log_file" (log verbose) creates a verbose installation log with the name you specified; if only a file name is provided it is created in the directory where the msiexec was launched.

    • /i "msi_file" points to the HDP Installer file, we recommend specifying the absolute path.

    • MSIUSEREALADMINDETECTION=1 ensures that the user running the installer has true administrator permissions.

    For more information on standard msiexec options, enter msiexec /? in a command prompt.

  • HDP public properties, that is everything following the last option /i "msi_file" in the command line format above, are described in the following table. These public properties are passed by msiexec to the Hadoop setup script:

     

    Table 4.1. HDP Public Properties

    PropertyValueValue Defined in Cluster Properties fileDescription
    DESTROY_DATAYes or NononeYes removes previous HDP data and formats the NameNode. No leaves the previous data and does not format the NameNode.
    HDP_USER_PASSWORDpasswordnonePassword defined when creating the Hadoop user. Note that if the password does not meet your password policy standards the installation will fail.
    HDP_DIRinstall_dirnoneAbsolute path to the Hadoop root directory where HDP components are installed.
    HDP_LAYOUTclusterproperties_full_pathnoneDefines the absolute path to the Cluster Properties file. Note that relative paths are not supported and the path may not contain spaces. For example, C:\MSI_Install\clusterproperties.txt .
    HDPYes or NoZOOKEEPER_HOSTSSetting this to Yes instructs the MSI to install the optional HDP components, such as Flume, HBase, Knox, Falcon and Storm. When enabled, you must specify the components on the commandline. For example: HDP="YES" KNOX="YES" KNOX_SECRET="secret" FALCON="NO" HBASE="YES" FLUME="NO" STORM="NO". Excluding the optional components from the command line causes the installation to fail.
    FLUMEYes or NoFLUME_HOSTSIncludes the installation of Flume components on the hosts matching the name defined in the cluster properties file.
    HBASEYes or NoHBASE_MASTER and HBASE_REGIONSERVERSIncludes the installation of HBase components on the hosts matching the name defined in the cluster properties file.
    KNOXYes or NoKNOX_HOSTIncludes the installation of Knox Gateway on the host matching the name defined in the cluster properties file. When yes the KNOX_SECRET must also be specified as a parameter.
    KNOX_MASTER_SECRETsecretnoneSpecified only when KNOX="yes". The master secret to protect Knox security components, such as SSL certificates.
    FALCONYes or NoFALCON_HOSTSIncludes the installation of the Falcon components on the host matching the name defined in the cluster properties file.
    STORMYes or NoSTORM_NIMBUS and STORM_SUPERVISORSIncludes the installation of the Storm components on the host matching the name defined in the cluster properties file.

For the optional HDP Components, specify the same property, such as HDP=yes FLUME=no HBASE=yes KNOX=no FALCON=no STORM=no, on the command line of every cluster node. If you are not installing any optional components specify HDP=no. Components are only installed if the hostname matches a value in the cluster properties file.

To install a basic cluster with HBase, use the following command on every node:

msiexec /qn /i  D:\MSI_Install\hdp-2.1.1.0.winpkg.msi /lv D:\MSI_Install\hdp.log MSIUSEREALADMINDETECTION=1 HDP_LAYOUT=D:\MSI_Install\cluster.properties HDP_DIR=D:\hdp DESTROY_DATA=yes HDP_USER_PASSWORD=#TestUser123! HDP=yes KNOX=no FALCON=no STORM=no HBase=yes STORM=no FLUME=no
[Important]Important

Use the same clusterproperties.txt file on every node in the cluster. When LZO compression is enabled you must also put the following three files i the same directory:

  • hadoop-lzo-0.4.19.2.1.1.0-1621.jar from the HDP for Windows Installation zip.

  • gplcompression.dll from the HDP for Windows Installation zip.

  • lzo2.dll LZO compression DLL downloaded from here.


loading table of contents...