DocsHortonworks Data Platform

 

 2. Add Hive Repositories

HDP Security Administration tools support access control and auditing for Hive repositories in Hadoop clusters.

 2.1. Create a Hive Repository

Before installing the agent on the HiveServer2 host set up a repository in the Policy Manager.

[Important]Important

For Hive connection information, see HiveServer2 Clients, JDBC.

To create a Hive Repository:

  1. Sign in to the HDP Security Administrator Web UI as an administrator.

  2. Click Policy Manager.

    The Manage Repository page displays.

  3. Next to Hive, click the green plus symbol.

    The Create Repository page displays.

  4. Complete the required settings with the following information:

     

    Table 4.4. Hive Repository Details

    LabelValueDescription
    Repository Name $nameSpecify a unique name for the repository, you will need to specify the repository name in the agent installation properties. For example, clustername_hive.
    Description$description-of-repoEnter a description up to 150 characters.
    Active StatusEnabled or DisabledEnable or disable policy enforcement for the repository.
    Repository typeHDFS, Hive, or HBaseSelect the type of repository, Hive.
    User name $userSpecify a user name on the remote system with permission to establish the connection with the hive, for example hive.
    Password$passwordSpecify the password of the user account for connection.
    jdbc.driverClassName$classnameSpecify the full classname of the driver used for Hive connections. The default HiveServer2 classname is org.apache.hive.jdbc.HiveDriver.
    jdbc.url$jdbc:hive2://hiveserver-host:port/dbSpecify the complete connection URL, including port (default port is 10000) and database name. For example on sandbox, jdbc:hive2://sandbox:10000/.


  5. Click Test Connection.

    If the server can establish a connection with HiveServer using the information you provided a success message displays.

  6. After the connection is successful, click Save.

 2.2. Install the Hive Agent on the HiveServer2 Host

After creating the Hive Repository in the Policy Manager, install the agent on the HiveServer2 host.

[Note]Note

If you are using Beeswax on Hue to run Hive queries, you must also install the Hive agent on the Hue server host.

 2.2.1. Installation Set Up

Perform the following steps on the HiveServer2 host.

  1. Log on to the host as root.

  2. Create a temporary directory, such as /tmp/xasecure:

    mkdir /tmp/xasecure

  3. Move the package into the temporary directory along with the MySQL Connector Jar.

  4. Extract the contents:

    tar xvf $xasecureinstallation.tar

  5. Go to the directory where you extracted the installation files:

    cd /tmp/xasecure/xasecure-$name-$build-version

  6. Open the install.properties file for editing.

  7. Change the following parameters for your environment:

     

    Table 4.5. Hive Agent Install Parameters

    ParameterValueDescription
    POLICY_MGR_URL$urlSpecify the full URL to access the Policy Manager Web UI. For example, http://pm-host:6080.
    MYSQL_CONNECTOR_JAR$path-to-mysql-connectorAbsolute path on the local host to the JDBC driver for mysql including filename.[a] For example, /tmp/xasecure/
    REPOSITORY_NAME$Policy-Manager-Repo-NameName of the HDFS Repository in the Policy Manager that this agent connects to after installation.
    XAAUDIT.DB.HOSTNAME$XAsecure-db-hostSpecify the host name of the MySQL database.
    XAAUDIT.DB.DATABASE_NAME$auditdbSpecify the audit database name that matches the audit_db_name specified during installation.
    XAAUDIT.DB.USER_NAME$auditdbuserSpecify the audit database name that matches the audit_db_user specified during installation
    XAAUDIT.DB.PASSWORD$auditdbupwSpecify the audit database name that matches the audit_db_password specified during installation

    [a] Download the JAR from here.


  8. Save the install.properties file.

[Note]Note

If your environment is configured to use SSL, modify the properties following the instructions in Set Up SSL for Hive Security Agent.

 2.2.2. Run the Agent Installation Script

After configuring the install.properties file, install the agent as root:

  1. Log on to the Linux system as root and go to the directory where you extracted the installation files:

    cd /tmp/xasecure/xasecure-$name-$build-version

  2. Run the agent installation script:

    # ./install.sh

 2.2.3. Restart the Hive Service

After installing the agent in an environment that does NOT have Ambari, manually restart the Hive services as follows:

  1. Stop Hive. Execute this command on the Hive Metastore and Hive Server2 host machine.

    ps aux | awk '{print $1,$2}' | grep hive | awk '{print $2}' | xargs kill >/dev/null 2>&1

  2. Start Hive Metastore. On the Hive Metastore host machine, execute the following command:

    su - hive -c "env HADOOP_HOME=/usr JAVA_HOME=/usr/jdk64/jdk1.6.0_31 /tmp/startMetastore.sh /var/log/hive/hive.out /var/log/hive/hive.log /var/run/hive/hive.pid /etc/hive/conf.server"

    where, $HIVE_LOG_DIR is the directory where Hive server logs are stored. For example, /var/logs/hive.

  3. Start HiveServer2. On the Hive Server2 host machine, execute the following command:

    su - hive -c "env JAVA_HOME=/usr/jdk64/jdk1.6.0_31 /tmp/startHiveserver2.sh /var/log/hive/hive-server2.out /var/log/hive/hive-server2.log /var/run/hive/hive-server.pid /etc/hive/conf.server"

    where $HIVE_LOG_DIR is the directory where Hive server logs are stored. For example, /var/logs/hive.

 2.3. Configure Hive in Ambari Environments

Follow the configuration steps in environments where Hive is managed by the Ambari Server:

 2.3.1. Modify the Ambari Hive Startup Script

Remove the HiveServer configuration string from the Ambari Hive startup script.

[Note]Note

Ambari starts and stops the HiveServer2 using a built in script. In order to start and stop HiveServer2 with the integrated Security Agent, you must comment out the HiveServer configuration string.

  1. Log into the Ambari Server Linux host using the Ambari account.

  2. Open the Ambari Hive startup script for editing:

    cd /var/lib/ambari-server/resources/stacks/HDP/2.0.6/services/HIVE/package/templates
vi startHiveserver2.sh.j2

  3. Comment out the following line by prepending a # at the beginning of the line as follows:

    # HIVE_SERVER2_OPTS="${HIVE_SERVER2_OPTS} –hiveconf hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator -hiveconf hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory"

  4. Restart the Ambari Server from the command line as follows:

    su -l ambari -c "/etc/init.d/ambari-server stop"
su -l ambari -c "/etc/init.d/ambari-server start"

  5. On each node in the cluster, restart the Ambari Agents:

    su -l ambari -c "/etc/init.d/ambari-agent stop"
su -l ambari -c "/etc/init.d/ambari-agent start"

After the Ambari Server and Agents finish rebooting, update the Hive Configuration with the required settings.

 2.3.2. Configure Hive

After changing the Ambari Hive startup script and restarting the Ambari Server from the command line, perform the following steps to configure Hive server for agent integration.

  1. Log into the Ambari Web UI, and click Hive > Config.

    [Note]Note

    To find a property, type the name in the Filter field and press enter; if the parameter exists, it is returned under the common or advanced list. Click the arrow key to expand the lists to see the settings.

  2. Update the following properties as follows:

    • Property name: hive.security.authorization.manager

      New Value: com.xasecure.authorization.hive.authorizer.XaSecureAuthorizer

    • Property name: hive.security.authorization.enabled

      New Value: true

  3. Filter for the hive.exec.pre.hooks property.

    Add the HDP Security hook after the existing value by inserting a comma followed by com.xasecure.authorization.hive.hooks.XaSecureHivePreExecuteRunHook.

    For example, if the existing value is org.apache.hadoop.hive.ql.hooks.ATSHook the new value with the HDP Security hook is: 

    org.apache.hadoop.hive.ql.hooks.ATSHook,com.xasecure.authorization.hive.hooks.XaSecureHivePreExecuteRunHook

  4. Search for the hive.exec.post.hooks property.

    Add the HDP Security hook after the existing value by inserting a comma followed by com.xasecure.authorization.hive.hooks.XaSecureHivePostExecuteRunHook.

    For example if the existing value is org.apache.hadoop.hive.ql.hooks.ATSHook the new value with the HDP Security hook is:

    org.apache.hadoop.hive.ql.hooks.ATSHook,com.xasecure.authorization.hive.hooks.XaSecureHivePostExecuteRunHook

  5. Expand Custom hive-site.xml, and add the following properties:

     

    Table 4.6. Custom hive-site.xml Properties

    KeyValue
    hive.semantic.analyzer.hookcom.xasecure.authorization.hive.hooks.XaSecureSemanticAnalyzerHook
    hive.server2.custom.authentication.classcom.xasecure.authentication.hive.LoginNameAuthenticator
    hive.conf.restricted.listhive.exec.driver.run.hooks, hive.server2.authentication, hive.metastore.pre.event.listeners, hive.security.authorization.enabled,hive.security.authorization.manager, hive.semantic.analyzer.hook, hive.exec.post.hooks


    [Note]Note

    For each property, click Add Property, enter Key and Value shown in the table above, then click Add.

  6. After all the properties have been updated and added, scroll to the bottom and click Save.

    The settings display under Custom hive-site.xml.

    When properties change, the affected services must be restarted. A Restart option displays.

  7. Click Restart > Restart all.

 2.4. Verify that Agent is Connected

Connected Agents display in the HDP Security Administration Web UI.

[Note]Note

Agents may not appear in the list until after the first event occurs in the repository.

To verify that the agent is connected to the server:

  1. Log in to the interface using the admin account.

  2. Click Audit > Agent.

Legal notices
loading table of contents...