Command Line Installation
Also available as:
PDF
loading table of contents...

Installing LLAP on an Unsecured Cluster

On the computer where Hive2 is installed, follow these steps:

  1. Log in as user hive:

    sudo su -
    su $HIVE_USER
  2. Generate the LLAP package:

    hive --service llap --name <llap_svc_name> --instances
    <number_of_cluster_nodes>
    --cache <cache_size>m --xmx <heap_size>m --size 
    ((<cache_size>+<heap_size>)*1.05)m
    --executors <number_of_cores> --loglevel <WARN|INFO> 
    --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA  -XX:-ResizePLAB"

    Table 9.3. 

    ParameterRecommended Value
    --instances <cache_size>Number of cluster nodes to use LLAP.
    --cache <cache_size>

    <YARN_maximum_container_size> - (<hive.tez.container.size> <number_of_cores>)

    <hive.tez.container.size> is the setting for this property, found in the hive-site.xml file. Depending on the size of the node, specify a minimum between 1 GB through 4 GB.

    --xmx <heap_size>

    For medium-sized nodes: <hive.tex.container.size> * <number of cores> * (0.8 to 0.95)

    <hive.tez.container.size> is the setting for this property found in the hive-site.xml file.

    Ensure that the setting for --xmx is 1 GB less than (<hive.tez.container.size> * <number_of_cores>). For smaller nodes, use the same formula, multiplied by 0.8.

    --executors <number_of_cores>Set to the number of CPU cores available on nodes running NodeManager. Set this value even if CPU scheduling is enabled in YARN.

  3. Set the --loglevel parameter to INFO when you are troubleshooting or testing.

    The INFO option provides verbose output. In a production environment, set the --loglevel parameter to WARN, which outputs a message to the logs only if there is a warning or error. This makes the logs easier to read and reduces load on the node.

    [Note]Note

    The recommended values listed in the previous table represent a sample configuration. LLAP also can be configured to use a fraction of node resources.

    The following message is returned, confirming that the LLAP package was created: Prepared llap-slider-<date>/run.sh for running LLAP on Slider

  4. Verify that the following LLAP and HiveServer2 properties are set in the hive-site.xml file, which can be located in the /etc/hive/conf/ or the /etc/hive/conf/conf.server/ directory:

    Table 9.4. LLAP Properties to Set in hive-site.xml

    PropertyValues
    hive.execution.modellap
    hive.llap.execution.modeall
    hive.llap.daemon.service.hosts@<llap_service_name>

    <llap_service_name> is the service name that you specified when you generated the LLAP package in Step 2.

    Table 9.5. HiveServer2 Properties to Set in hive-site.xml to Enable Concurrent Queries with LLAP

    PropertyValues
    hive.server2.tez.default.queuesQueue to use for the LLAP sessions. These correspond to YARN queues of the same name. For example, default.
    hive.server2.tez.initialize.default.sessionsTrue
    hive.server2.tez.sessions.per.default.queueSet to the number of concurrent queries to run on the queues that are configured by hive.server2.tez.default.queues. This setting launches long-running Tez AMs (sessions).

  5. Example of these properties set in the hive-site.xml file:

    <property>
         <name>hive.execution.mode</name>
         <value>llap</value>
    </property>
    
    <property>
         <name>hive.llap.execution.mode</name>
         <value>ALL</value>
    </property>
    
    <property>
         <name>hive.llap.daemon.service.hosts</name>
         <value>@<llap_service_name></value>
    </property>
    
    <property>
         <name>hive.server2.tez.default.queues</name>
         <value>default</value>
    </property>
    
    <property>
         <name>hive.server2.tez.initialize.default.sessions</name>
         <value>true</value>
    </property>
    
    <property>
         <name>hive.server2.tez.sessions.per.default.queue</name>
         <value>1</value>
    </property>