3. Enable Tez for Hive Queries

Use the following instructions to submit Hive queries to Tez:

  1. Copy the hive-exec-0.11.0.jar to HDFS at the following location: /apps/hive/install/hive-exec-0.11.0.jar.

    su - $HIVE_USER
    hadoop fs -mkdir /apps/hive/install
    hadoop fs -copyFromLocal /usr/lib/hive/lib/hive-exec-* /apps/hive/install/hive-exec-0.11.0.jar
  2. Execute the following command from the Hive client machines, to run Hive in MRR/Tez mode:

    export HIVE_AUX_JARS_PATH="/usr/lib/hcatalog/share/hcatalog/hcatalog-core.jar:$TEZ_CONF_DIR:$TEZ_INSTALL_DIR/*:$TEZ_INSTALL_DIR/lib/*"

    where

    • $TEZ_CONF_DIR is the directory for storing the Tez configuration files. For example, /etc/tez/conf.

    • $TEZ_INSTALL_DIR is the directory where Tez is installed. For example, /usr/lib/tez/.

  3. Enable Hive to use Tez DAG APIs. On the Hive client machine, add the following to your Hive script or execute the following in the hive shell:

    set hive.use.tez.natively=true;

  4. Allow Hive to combine multipe mapreduce (MR) jobs into a single MRR job where possible:

    set hive.enable.mrr=true;


loading table of contents...