Non-Ambari Cluster Installation Guide
Also available as:
PDF
loading table of contents...

Installing Ranger Plug-ins

The following sections describe how to install Ranger plug-ins.

[Note]Note

To ensure that you are installing the HDP version of the plug-ins instead of the Apache version, make sure you enter the following commands when installing each plug-in:

  • For CentOS and RHEL:

    yum install ranger_ <version_number>
  • For SLES:

    zypper -n --no-gpg-checks install --auto-agree-with-licenses ranger_ <version_number>
  • For Debian/Ubuntu:

    HDP support for Debian 6 is deprecated with HDP 2.3.6. Future versions of HDP will no longer be supported on Debian 6.

    apt-get install <version_number>

Installing the Ranger HDFS Plug-in

The Ranger HDFS plug-in helps to centralize HDFS authorization policies.

This section describes how to create an HDFS repository and install the HDFS plug-in.

Install the HDFS Plug-in

  1. Create a Ranger HDFS policy using the instructions in Create an HDFS Policy in the Hadoop Security Guide.

    Make a note of the name you gave to this repository; you will need to use it again during HDFS plug-in setup.

  2. At all servers where NameNode is installed, install the HDFS plug-in by following the steps listed below:

    1. Go to the home directory of the HDFS plug-in:

      cd /usr/hdp/<version>/ranger-hdfs-plugin
    2. Edit the following HDFS-related properties in the install.properties file:

      Table 13.3. Properties to Edit in the install.properties File

      Configuration Property Name

      Default/Example Value

      Required?

      Policy Admin Tool

      POLICY_MGR_URL URL for policy admin

      http://policymanager.xasecure.net:6080

      Y

      REPOSITORY_NAME The repository name used in Policy Admin Tool for defining policies

      hadoopdev

      Y

      Audit Database

      SQL_CONNECTOR_JAR Path to SQL connector jar of the DB Flavor selected. The value should be the absolute path including the jar name.

      /usr/share/java/mysql-connector-java.jar (default)

      /usr/share/java/postgresql.jar

      /usr/share/java/sqljdbc4.jar

      /usr/share/java/ojdbc6.jar

      Y

      XAAUDIT.DB.IS_ENABLEDEnable or disable database audit logging.

      FALSE (default), TRUE

      Y

      XAAUDIT.DB.FLAVOUR Specifies the type of database used for audit logging (MYSQL,ORACLE)

      MYSQL (default)

      Y

      XAAUDIT.DB.HOSTNAME Hostname of the audit database server

      localhost

      Y

      XAAUDIT.DB.DATABASE_NAME Audit database name

      ranger_audit

      Y

      XAAUDIT.DB.USER_NAME Username used for performing audit log inserts (should be same username used in the ranger-admin installation process)

      rangerlogger

      Y

      XAAUDIT.DB.PASSWORD Database password associated with the above database user - for db audit logging

      rangerlogger

      Y

      HDFS Audit

      XAAUDIT.HDFS.IS_ENABLED Flag to enable/disable hdfs audit logging. If the hdfs audit logging is turned off, it will not log any access control to hdfs

      Y

      XAAUDIT.HDFS.DESTINATION _DIRECTORY HDFS directory where the audit log will be stored

      hdfs://__REPLACE__NAME_NODE_HOST:8020/ (format) hdfs://namenode.mycompany.com:8020/ranger/audit/%app-type%/%time:yyyyMMdd%

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _DIRECTORY Local directory where the audit log will be saved for intermediate storage

      hdfs://__REPLACE__NAME_NODE_HOST:8020/ (format) /var/log/%app-type%/audit

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _DIRECTORY Local directory where the audit log will be archived after it is moved to hdfs

      __REPLACE__LOG_DIR%app-type%/audit/archive (format) /var/log/%app-type%/audit/archive

      Y

      XAAUDIT.HDFS.DESTINATION_FILE hdfs audit file name (format)

      %hostname%-audit.log (default)

      Y

      XAAUDIT.HDFS.DESTINATION _FLUSH_INTERVAL_SECONDS hdfs audit log file writes are flushed to HDFS at regular flush interval

      900

      Y

      XAAUDIT.HDFS.DESTINATION _ROLLOVER_INTERVAL_SECONDS hdfs audit log file is rotated to write to a new file at a rollover interval specified here

      86400

      Y

      XAAUDIT.HDFS.DESTINATION _OPEN_RETRY_INTERVAL_SECONDS hdfs audit log open() call is failed, it will be re-tried at this interval

      60

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FILE Local filename used to store in audit log (format)

      %time:yyyyMMdd-HHmm.ss%.log (default)

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FLUSH_INTERVAL_SECONDS Local audit log file writes are flushed to filesystem at regular flush interval

      60

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _ROLLOVER_INTERVAL_SECONDS Local audit log file is rotated to write to a new file at a rollover interval specified here

      600

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _MAX_FILE_COUNT The maximum number of local audit log files that will be kept in the archive directory

      10

      Y

      SSL Information (https connectivity to Policy Admin Tool)

      SSL_KEYSTORE_FILE_PATH Java Keystore Path where SSL key for the plug-in is stored. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

      /etc/hadoop/conf/ranger-plugin-keystore.jks (default)

      Only if SSL is enabled

      SSL_KEYSTORE_PASSWORD Password associated with SSL Keystore. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

      none (default)

      Only if SSL is enabled

      SSL_TRUSTSTORE_FILE_PATH Java Keystore Path where the trusted certificates are stored for verifying SSL connection to Policy Admin Tool. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

      /etc/hadoop/conf/ranger-plugin-truststore.jks (default)

      Only if SSL is enabled

      SSL_TRUSTSTORE_PASSWORD Password associated with Truststore file. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

      none (default)

      Only if SSL is enabled


  3. To enable the HDFS plug-in, run the following commands:

    cd /usr/hdp/<version>/ranger-hdfs-plugin
    ./enable-hdfs-plugin.sh
  4. To confirm that installation and configuration are complete, go to the Audit Tab of the Ranger Admin Console and check Plugins. You should see HDFS listed there.

Installing the Ranger YARN Plug-in

This section describes how to install and enable the Ranger YARN plug-in.

  1. The Ranger YARN plug-in is automatically installed when YARN is installed. You can verify this plug-in is present by using the following command:

    rpm -qa | grep yarn-plugin
    ranger_2_3_6_0_$BUILD-yarn-plugin-0.5.0.2.3.6.0-$BUILD.el6.x86_64
  2. Navigate to /usr/hdp/<version>/ranger-yarn-plugin.

    cd /usr/hdp/<version>/ranger-yarn-plugin
  3. Edit the following entries in the install.properties file.

    Table 13.4. Properties to Edit in the install.properties File

    Configuration Property Name

    Default/Example Value

    Required?

    Policy Admin Tool

    POLICY_MGR_URL URL for policy admin

    http://<FQDN of ranger admin host>:6080

    Y

    REPOSITORY_NAME The repository name used in Policy Admin Tool for defining policies

    yarndev

    Y

    Audit Database

    SQL_CONNECTOR_JAR Path to SQL connector jar of the DB Flavor selected. The value should be the absolute path including the jar name.

    /usr/share/java/mysql-connector-java.jar (default)

    /usr/share/java/postgresql.jar

    /usr/share/java/sqljdbc4.jar

    /usr/share/java/ojdbc6.jar

    Y

    XAAUDIT.DB.IS_ENABLEDEnable or disable database audit logging.

    FALSE (default), TRUE

    Y

    XAAUDIT.DB.FLAVOUR Specifies the type of database used for audit logging (MYSQL,ORACLE)

    MYSQL (default)

    Y

    XAAUDIT.DB.HOSTNAME Hostname of the audit database server

    localhost

    Y

    XAAUDIT.DB.DATABASE_NAME Audit database name

    ranger_audit

    Y

    XAAUDIT.DB.USER_NAME Username used for performing audit log inserts (should be same username used in the ranger-admin installation process)

    rangerlogger

    Y

    XAAUDIT.DB.PASSWORD Database password associated with the above database user - for db audit logging

    rangerlogger

    Y

    HDFS Audit

    XAAUDIT.HDFS.IS_ENABLED Flag to enable/disable hdfs audit logging. If the hdfs audit logging is turned off, it will not log any access control to hdfs

    Y

    XAAUDIT.HDFS.DESTINATION _DIRECTORY HDFS directory where the audit log will be stored

    hdfs://__REPLACE__NAME_NODE_HOST:8020/ (format) hdfs://namenode.mycompany.com:8020/ranger/audit/%app-type%/%time:yyyyMMdd%

    Y

    XAAUDIT.HDFS.LOCAL_BUFFER _DIRECTORY Local directory where the audit log will be saved for intermediate storage

    hdfs://__REPLACE__NAME_NODE_HOST:8020/ (format) /var/log/%app-type%/audit

    Y

    XAAUDIT.HDFS.LOCAL_ARCHIVE _DIRECTORY Local directory where the audit log will be archived after it is moved to hdfs

    __REPLACE__LOG_DIR%app-type%/audit/archive (format) /var/log/%app-type%/audit/archive

    Y

    XAAUDIT.HDFS.DESTINATION_FILE hdfs audit file name (format)

    %hostname%-audit.log (default)

    Y

    XAAUDIT.HDFS.DESTINATION _FLUSH_INTERVAL_SECONDS hdfs audit log file writes are flushed to HDFS at regular flush interval

    900

    Y

    XAAUDIT.HDFS.DESTINATION _ROLLOVER_INTERVAL_SECONDS hdfs audit log file is rotated to write to a new file at a rollover interval specified here

    86400

    Y

    XAAUDIT.HDFS.DESTINATION _OPEN_RETRY_INTERVAL_SECONDS hdfs audit log open() call is failed, it will be re-tried at this interval

    60

    Y

    XAAUDIT.HDFS.LOCAL_BUFFER _FILE Local filename used to store in audit log (format)

    %time:yyyyMMdd-HHmm.ss%.log (default)

    Y

    XAAUDIT.HDFS.LOCAL_BUFFER _FLUSH_INTERVAL_SECONDS Local audit log file writes are flushed to filesystem at regular flush interval

    60

    Y

    XAAUDIT.HDFS.LOCAL_BUFFER _ROLLOVER_INTERVAL_SECONDS Local audit log file is rotated to write to a new file at a rollover interval specified here

    600

    Y

    XAAUDIT.HDFS.LOCAL_ARCHIVE _MAX_FILE_COUNT The maximum number of local audit log files that will be kept in the archive directory

    10

    Y

    SSL Information (https connectivity to Policy Admin Tool)

    SSL_KEYSTORE_FILE_PATH Java Keystore Path where SSL key for the plug-in is stored. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

    /etc/hadoop/conf/ranger-plugin-keystore.jks (default)

    Only if SSL is enabled

    SSL_KEYSTORE_PASSWORD Password associated with SSL Keystore. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

    none (default)

    Only if SSL is enabled

    SSL_TRUSTSTORE_FILE_PATH Java Keystore Path where the trusted certificates are stored for verifying SSL connection to Policy Admin Tool. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

    /etc/hadoop/conf/ranger-plugin-truststore.jks (default)

    Only if SSL is enabled

    SSL_TRUSTSTORE_PASSWORD Password associated with Truststore file. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

    none (default)

    Only if SSL is enabled


  4. Enable the YARN plug-in by running the following commands:

    export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64

    or

    export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64

    ./enable-yarn-plugin.sh

  5. Make sure HADOOP_YARN_HOME and HADOOP_LIBEXEC_DIR are set.

    export HADOOP_YARN_HOME=/usr/hdp/current/hadoop-yarn-nodemanager/
    export HADOOP_LIBEXEC_DIR=/usr/hdp/current/hadoop-client/libexec/
    
  6. Enter the following commands to stop/start the ResourceManager on all of your Resource Manager hosts.

    su yarn -c "/usr/hdp/current/hadoop-yarn-resourcemanager/sbin/yarn-daemon.sh stop resourcemanager"   
    su yarn -c "/usr/hdp/current/hadoop-yarn-resourcemanager/sbin/yarn-daemon.sh start resourcemanager"        
    ps -ef | grep -i resourcemanager
    
  7. Enter the following command to stop/start the NodeManager on all of your NodeManager hosts.

    su yarn -c "/usr/hdp/current/hadoop-yarn-nodemanager/sbin/yarn-daemon.sh stop nodemanager"
    su yarn -c "/usr/hdp/current/hadoop-yarn-nodemanager/sbin/yarn-daemon.sh start nodemanager"
    ps -ef | grep -i nodemanager
    
  8. Create the default repo for YARN with the proper configuration specifying the same repository name as in step 3.

  9. You can verify the plug-in is communicating to Ranger admin via the Audit/plugins tab.

  10. Optional: If Wire Encryption is enabled after Ranger authorization is set up for YARN, follow these steps to update the resource manager URL in Ranger:

    1. Login to Ranger Admin as an admin user.

    2. Click 'Edit' for the YARN service. The service name is something like cl1_yarn (where 'cl1' is the name of the cluster).

    3. Update property 'YARN REST URL', with the https URL for the resource manager.

    4. Click 'Save.'

Installing the Ranger Kafka Plug-in

This section describes how to install and enable the Ranger Kafka plug-in.

  1. The Ranger Kafka plug-in is automatically installed when Kafka is installed. You can verify this plug-in is present by using the following command:

    rpm -qa | grep kafka-plugin
    ranger_2_3_6_0_$BUILD-kafka-plugin-0.5.0.2.3.6.0-$BUILD.el6.x86_64
  2. Navigate to /usr/hdp/<version>/ranger-kafka-plugin.

    cd /usr/hdp/<version>/ranger-kafka-plugin
  3. Edit the following entries in the install.properties file.

    Table 13.5. Properties to Edit in the install.properties File

    Configuration Property Name

    Default/Example Value

    Required?

    Policy Admin Tool

    COMPONENT_INSTALL_DIR_NAME/usr/hdp/2.3.6.0-$BUILD/kafka Y

    POLICY_MGR_URL URL for policy admin

    http://<FQDN of ranger admin host>:6080

    Y

    REPOSITORY_NAME The repository name used in Policy Admin Tool for defining policies

    kafkadev

    Y

    Audit Database

    SQL_CONNECTOR_JAR Path to SQL connector jar of the DB Flavor selected. The value should be the absolute path including the jar name.

    /usr/share/java/mysql-connector-java.jar (default)

    /usr/share/java/postgresql.jar

    /usr/share/java/sqljdbc4.jar

    /usr/share/java/ojdbc6.jar

    Y

    XAAUDIT.DB.IS_ENABLEDEnable or disable database audit logging.

    FALSE (default), TRUE

    Y

    XAAUDIT.DB.FLAVOUR Specifies the type of database used for audit logging (MYSQL,ORACLE)

    MYSQL (default)

    Y

    XAAUDIT.DB.HOSTNAME Hostname of the audit database server

    localhost

    Y

    XAAUDIT.DB.DATABASE_NAME Audit database name

    ranger_audit

    Y

    XAAUDIT.DB.USER_NAME Username used for performing audit log inserts (should be same username used in the ranger-admin installation process)

    rangerlogger

    Y

    XAAUDIT.DB.PASSWORD Database password associated with the above database user - for db audit logging

    rangerlogger

    Y

    HDFS Audit

    XAAUDIT.HDFS.IS_ENABLED Flag to enable/disable hdfs audit logging. If the hdfs audit logging is turned off, it will not log any access control to hdfs

    Y

    XAAUDIT.HDFS.DESTINATION _DIRECTORY HDFS directory where the audit log will be stored

    hdfs://__REPLACE__NAME_NODE_HOST:8020/ (format) hdfs://namenode.mycompany.com:8020/ranger/audit/%app-type%/%time:yyyyMMdd%

    Y

    XAAUDIT.HDFS.LOCAL_BUFFER _DIRECTORY Local directory where the audit log will be saved for intermediate storage

    hdfs://__REPLACE__NAME_NODE_HOST:8020/ (format) /var/log/%app-type%/audit

    Y

    XAAUDIT.HDFS.LOCAL_ARCHIVE _DIRECTORY Local directory where the audit log will be archived after it is moved to hdfs

    __REPLACE__LOG_DIR%app-type%/audit/archive (format) /var/log/%app-type%/audit/archive

    Y

    XAAUDIT.HDFS.DESTINATION_FILE hdfs audit file name (format)

    %hostname%-audit.log (default)

    Y

    XAAUDIT.HDFS.DESTINATION _FLUSH_INTERVAL_SECONDS hdfs audit log file writes are flushed to HDFS at regular flush interval

    900

    Y

    XAAUDIT.HDFS.DESTINATION _ROLLOVER_INTERVAL_SECONDS hdfs audit log file is rotated to write to a new file at a rollover interval specified here

    86400

    Y

    XAAUDIT.HDFS.DESTINATION _OPEN_RETRY_INTERVAL_SECONDS hdfs audit log open() call is failed, it will be re-tried at this interval

    60

    Y

    XAAUDIT.HDFS.LOCAL_BUFFER _FILE Local filename used to store in audit log (format)

    %time:yyyyMMdd-HHmm.ss%.log (default)

    Y

    XAAUDIT.HDFS.LOCAL_BUFFER _FLUSH_INTERVAL_SECONDS Local audit log file writes are flushed to filesystem at regular flush interval

    60

    Y

    XAAUDIT.HDFS.LOCAL_BUFFER _ROLLOVER_INTERVAL_SECONDS Local audit log file is rotated to write to a new file at a rollover interval specified here

    600

    Y

    XAAUDIT.HDFS.LOCAL_ARCHIVE _MAX_FILE_COUNT The maximum number of local audit log files that will be kept in the archive directory

    10

    Y

    SSL Information (https connectivity to Policy Admin Tool)

    SSL_KEYSTORE_FILE_PATH Java Keystore Path where SSL key for the plug-in is stored. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

    /etc/hadoop/conf/ranger-plugin-keystore.jks (default)

    Only if SSL is enabled

    SSL_KEYSTORE_PASSWORD Password associated with SSL Keystore. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

    none (default)

    Only if SSL is enabled

    SSL_TRUSTSTORE_FILE_PATH Java Keystore Path where the trusted certificates are stored for verifying SSL connection to Policy Admin Tool. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

    /etc/hadoop/conf/ranger-plugin-truststore.jks (default)

    Only if SSL is enabled

    SSL_TRUSTSTORE_PASSWORD Password associated with Truststore file. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used

    none (default)

    Only if SSL is enabled


  4. Enable the Kafka plug-in by running the following commands:

    export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64
    ./enable-kafka-plugin.sh
  5. Enter the following commands to stop/start the Kafka service.

    su kafka -c "/usr/hdp/current/kafka-broker/bin/kafka stop" 
    su kafka -c "/usr/hdp/current/kafka-broker/bin/kafka start"
    
  6. Create the default repo for Kafka with the proper configuration specifying the same repository name as in step 3.

  7. You can verify the plug-in is communicating to Ranger admin via the Audit/plugins tab.

  8. If the plug-in is not able to communicate with Ranger admin, check the property authorizer.class.name in /usr/hdp/2.3.6.0-$BUILD/kafka/config/server.properties. The value of the authorizer.class.name should be org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.

Installing the Ranger HBase Plug-in

The Ranger HBase Plug-in integrates with HBase to enforce authorization policies.

This section describes how to install the HBase plug-in:

  1. Create an HBase repository

  2. Install the HBase plug-in and configure related HBase properties

  3. Enable the HBase plug-in

  4. Restart HBase

Install the HBase Plug-in

  1. Create a Ranger HDFS policy using the instructions in Create an HDFS Policy in the Hadoop Security Guide.

    Make a note of the name you gave to this repository; you will need to use it again during HDFS plug-in setup.

  2. At all servers where the HBase Master and RegionServers are installed, install and configure the HBase plug-in, as follows:

    1. Go to the home directory of the HBase plug-in:

      cd /usr/hdp/<version>/ranger-hbase-plugin
    2. Edit the following HBase-related properties in the install.properties file:

      Table 13.6. HBase Properties to Edit in the install.properties file

      Configuration Property Name

      Default/Example Value

      Required?

      Policy Admin Tool

      POLICY_MGR_URL URL for policy admin

      http://policymanager.xasecure.net:6080

      Y

      REPOSITORY_NAME The repository name used in Policy Admin Tool for defining policies

      hbasedev

      Y

      Audit Database

      SQL_CONNECTOR_JAR Path to SQL connector jar of the DB Flavor selected. The value should be the absolute path including the jar name.

      /usr/share/java/mysql-connector-java.jar (default)

      /usr/share/java/postgresql.jar

      /usr/share/java/sqljdbc4.jar

      /usr/share/java/ojdbc6.jar

      Y

      XAAUDIT.DB.IS_ENABLEDEnable or disable database audit logging.

      Note: If this property is set to FALSE, Ranger will not store audit logs in the audit DB, and audit logs will not be visible in the Ranger UI. If you would like to access audit logs from the UI, set this value to TRUE.

      FALSE (default)

      Y

      XAAUDIT.DB.FLAVOUR Specifies the type of database used for audit logging (MYSQL,ORACLE)

      MYSQL (default)

      Y

      XAAUDIT.DB.HOSTNAME Hostname of the audit database server

      localhost

      Y

      XAAUDIT.DB.DATABASE_NAME Audit database name

      ranger_audit

      Y

      XAAUDIT.DB.USER_NAME Username used for performing audit log inserts (should be same username used in the ranger-admin installation process)

      rangerlogger

      Y

      XAAUDIT.DB.PASSWORD Database password associated with the above database user - for db audit logging

      rangerlogger

      Y

      HDFS Audit

      XAAUDIT.HDFS.IS_ENABLED Flag to enable/disable hdfs audit logging. If the hdfs audit logging is turned off, it will not log any access control to hdfs

      TRUE

      Y

      XAAUDIT.HDFS.DESTINATION _DIRECTORY HDFS directory where the audit log will be stored

      hdfs://__REPLACE__NAME_NODE_HOST:8020/ranger/audit/%app-type%/%time:yyyyMMdd% (format) hdfs://namenode.mycompany.com:8020/ranger/audit/%app-type%/%time:yyyyMMdd%

      Y

      XAAUDIT.HDFS.LOCAL _BUFFER_DIRECTORY Local directory where the audit log will be saved for intermediate storage

      __REPLACE__LOG_DIR/%app-type%/audit (format) /var/tmp/%app-type%/audit

      Y

      XAAUDIT.HDFS.LOCAL _ARCHIVE_DIRECTORY Local directory where the audit log will be archived after it is moved to hdfs

      __REPLACE__LOG_DIR/%app-type%/audit/archive (format) /var/tmp/%app-type%/audit/archive

      Y

      XAAUDIT.HDFS.DESTINATION_FILE HDFS audit file name (format)

      %hostname%-audit.log (default)

      Y

      XAAUDIT.HDFS.DESTINATION _FLUSH_INTERVAL_SECONDS HDFS audit log file writes are flushed to HDFS at regular flush interval

      900

      Y

      XAAUDIT.HDFS.DESTINATION _ROLLOVER_INTERVAL_SECONDS HDFS audit log file is rotated to write to a new file at a rollover interval specified here

      86400

      Y

      XAAUDIT.HDFS.DESTINATION _OPEN_RETRY_INTERVAL_SECONDS If HDSF audit log open() call fails, it will be re-tried at this interval

      60

      Y

      XAAUDIT.HDFS.LOCAL _BUFFER_FILE Local filename used to store in audit log (format)

      %time:yyyyMMdd-HHmm.ss%.log (default)

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FLUSH_INTERVAL_SECONDS Interval that local audit log file writes are flushed to filesystem

      60

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _ROLLOVER_INTERVAL_SECONDS Interval that local audit log file is rolled over (rotated to write to a new file)

      600

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _MAX_FILE_COUNT The maximum number of local audit log files will be kept in the archive directory

      10

      Y

      SSL_KEYSTORE_FILE_PATH Java Keystore Path where SSL key for the plug-in is stored. Used only if SSL is enabled between Policy Admin Tool and Plugin. If SSL is not enabled, leave the default value as it is (should not be set as EMPTY).

      /etc/hbase/conf/ranger-plugin-keystore.jks (default)

      Y, if SSL is enabled

      SSL_KEYSTORE_PASSWORD Password associated with SSL Keystore. Used only if SSL is enabled between Policy Admin Tool and Plugin. If SSL is not Enabled, leave the default value as it is (should not be set as EMPTY).

      myKeyFilePassword (default)

      Y, if SSL is enabled

      SSL_TRUSTSTORE_FILE_PATH Java Keystore Path where the trusted certificates are stored for verifying SSL connection to Policy Admin Tool. Used only if SSL is enabled between Policy Admin Tool and Plugin. If SSL is not enabled, leave the default value as it is (should not be set as EMPTY).

      /etc/hbase/conf/ranger-plugin-truststore.jks (default)

      Y, if SSL is enabled

      SSL_TRUSTSTORE_PASSWORD Password associated with Truststore file. Used only if SSL is enabled between Policy Admin Tool and Plugin. If SSL is not Enabled, leave the default value as it is (should not be set as EMPTY).

      changeit (default)

      Y, if SSL is enabled

      HBase GRANT/REVOKE Commands

      UPDATE_XAPOLICIES_ON_GRANT_RE VOKE Provide ability for XAAgent to update the policies based on the GRANT/REVOKE commands from the HBase client

      TRUE (default)

      Y


  3. To enable the HBase plug-in, enter the following commands:

    cd /usr/hdp/<version>l/ranger-hbase-plugin
    ./enable-hbase-plugin.sh
  4. Restart HBase.

  5. To confirm that the HBase plug-in installation and configuration are complete, go to the Audit Tab of the Ranger Admin Console and check Plugins. You should see HBase listed there.

Installing the Ranger Hive Plug-in

The Ranger Hive plug-in integrates with Hive to enforce authorization policies.

[Note]Note

The Ranger plugin for HIve only needs to be set up for HiveServer2. For Hive clients, it is recommended that you protect data using HDFS policies in Ranger. Do not install or set up Ranger plugins on individual Hive client machines.

This section describes how to install the Ranger Hive plug-in:

  1. Create a Hive repository.

  2. Install the Hive plug-in and configure related Hive properties.

  3. Enable the Hive plug-in.

  4. Restart Hive.

Install the Hive Plug-in

  1. Create a Ranger HDFS policy using the instructions in Create an HDFS Policy in the Hadoop Security Guide.

    Make a note of the name you gave to this repository; you will need to use it again during HDFS plug-in setup.

  2. At the server where HiveServer2 is installed, install the Hive plug-in:

    • Go to the home directory of the Hive plug-in:

      cd /usr/hdp/<version>/ranger-hive-plugin
    • Edit the following Hive-related properties in the install.properties file:

      Table 13.7. Hive-Related Properties to Edit in the install.properties File

      Configuration Property Name

      Default/Example Value

      Required?

      Policy Admin Tool

      POLICY_MGR_URL URL for policy admin

      http://policymanager.xasecure.net:6080

      Y

      REPOSITORY_NAME The repository name used in Policy Admin Tool for defining policies

      hivedev

      Y

      Audit Database

      SQL_CONNECTOR_JAR Path to SQL connector jar of the DB Flavor selected. The value should be the absolute path including the jar name.

      /usr/share/java/mysql-connector-java.jar (default)

      /usr/share/java/postgresql.jar

      /usr/share/java/sqljdbc4.jar

      /usr/share/java/ojdbc6.jar

      Y

      XAAUDIT.DB.IS_ENABLED Enable or disable database audit logging.

      Note: If this property is set to FALSE, Ranger will not store audit logs in the audit DB, and audit logs will not be visible in the Ranger UI. If you would like to access audit logs from the UI, set this value to TRUE.

      FALSE (default) TRUE

      Y

      XAAUDIT.DB.FLAVOUR Specifies the type of database used for audit logging (MYSQL,ORACLE)

      MYSQL (default)

      Y

      XAAUDIT.DB.HOSTNAME Hostname of the audit database server

      localhost

      Y

      XAAUDIT.DB.DATABASE_NAME Audit database name

      ranger_audit

      Y

      XAAUDIT.DB.USER_NAME Username used for performing audit log inserts (should be same username used in the ranger-admin installation process)

      rangerlogger

      Y

      XAAUDIT.DB.PASSWORD database password associated with the above database user - for db audit logging

      rangerlogger

      Y

      HDFS Audit

      XAAUDIT.HDFS.IS_ENABLED Flag to enable/disable hdfs audit logging. If the hdfs audit logging is turned off, it will not log any access control to hdfs

      Y

      XAAUDIT.HDFS.DESTINATION _DIRECTORY HDFS directory where the audit log will be stored

      hdfs://__REPLACE__NAME_NODE_HOST:8020/ranger/audit/%app-type%/%time:yyyyMMdd% (format)

      hdfs://namenode.mycompany.com:8020/ranger/audit/%app-type%/%time:yyyyMMdd%

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _DIRECTORY Local directory where the audit log will be saved for intermediate storage

      __REPLACE__LOG_DIR/%app-type%/audit (format) /var/tmp/%app-type%/audit

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _DIRECTORY Local directory where the audit log will be archived after it is moved to hdfs

      __REPLACE__LOG_DIR/%app-type%/audit (format) /var/tmp/%app-type%/audit/archive

      Y

      XAAUDIT.HDFS.DESTINATION_FILE hdfs audit file name (format)

      %hostname%-audit.log (default)

      Y

      XAAUDIT.HDFS.DESTINATION _FLUSH_INTERVAL_SECONDS hdfs audit log file writes are flushed to HDFS at regular flush interval

      900

      Y

      XAAUDIT.HDFS.DESTINATION _ROLLOVER_INTERVAL_SECONDS hdfs audit log file is rotated to write to a new file at a rollover interval specified here

      86400

      Y

      XAAUDIT.HDFS.DESTINATION _OPEN_RETRY_INTERVAL_SECONDS If hdfs audit log open() call is failed, it will be re-tried at this interval

      60

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FILE Local filename used to store in audit log (format)

      %time:yyyyMMdd-HHmm.ss%.log (default)

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FLUSH_INTERVAL_SECONDS Local audit log file writes are flushed to filesystem at regular flush interval

      60

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _ROLLOVER_INTERVAL_SECONDS Local audit log file is rotated to write to a new file at a rollover interval specified here

      600

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _MAX_FILE_COUNT The maximum number of local audit log files that will be kept in the archive directory

      10

      Y

      SSL Information (https connectivity to Policy Admin Tool)

      SSL_KEYSTORE_FILE_PATH Java Keystore Path where SSL key for the plug-in is stored. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      /etc/hive/conf/ranger-plugin-keystore.jks (default)

      If SSL is enabled

      SSL_KEYSTORE_PASSWORD Password associated with SSL Keystore. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      none (default)

      If SSL is enabled

      SSL_TRUSTSTORE_FILE_PATH Java Keystore Path where the trusted certificates are stored for verifying SSL connection to Policy Admin Tool. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      /etc/hive/conf/ranger-plugin-truststore.jks (default)

      If SSL is enabled

      SSL_TRUSTSTORE_PASSWORD Password associated with Truststore file. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      none (default)

      If SSL is enabled

      Hive GRANT/REVOKE Command Handling

      UPDATE_XAPOLICIES_ON_GRANT _REVOKE Provide ability for XAAgent to update the policies based on the grant/revoke commands from the Hive beeline client

      TRUE (default)

      Y


  3. To enable the Hive plug-in, enter the following commands:

    cd /usr/hdp/<version>/ranger-hive-plugin
    ./enable-hive-plugin.sh
  4. Restart Hive.

  5. To confirm that the Hive plug-in installation and configuration are complete, go to the Audit Tab of the Ranger Admin Console and check Plugins. You should see Hive listed there.

Installing the Ranger Knox Plug-in

The Ranger Knox plug-in integrates with Knox to enforce authorization policies.

This section describes how to install the Knox plug-in:

  1. Create a Knox repository.

  2. Install the Knox plug-in and configure related Hive properties.

  3. Enable the Knox plug-in.

  4. Restart Knox.

Instructions assume that Knox has already been installed, as described in "Installing Knox."

Install the Knox Plug-in

  1. Create a Ranger HDFS policy using the instructions in Create an HDFS Policy in the Hadoop Security Guide.

    Make a note of the name you gave to this repository; you will need to use it again during HDFS plug-in setup.

  2. Set the URL to https://knox_host:8443/gateway/admin/api/v1/topologies, where knox_host is the full-qualified name of your Knox host machine.

  3. Make a note of the name you gave to this repository; you will need to use it again during Knox plug-in setup.

  4. At all servers where Knox Gateway is installed, install the Knox plug-in:

    1. Go to the home directory of the Knox plug-in:

      cd /usr/hdp/<version>/ranger-knox-plugin
    2. Edit the following Knox-related properties in the install.properties file:

      Table 13.8. Knox-Related Properties to Edit in the install.properties File

      Configuration Property Name

      Default/Example Value

      Mandatory?

      Policy Admin Tool

       

       

      POLICY_MGR_URL URL for policy admin

      http://policymanager.xasecure.net:6080

      Y

      REPOSITORY_NAME The repository name used in Policy Admin Tool for defining policies

      knoxdev

      Y

      Knox Component Installation

       

       

      KNOX_HOME Home directory where Knox software is installed

      /usr/hdp/current/knox

      Y

      Audit Database

       

       

      SQL_CONNECTOR_JAR Path to SQL connector jar of the DB Flavor selected. The value should be the absolute path including the jar name.

      /usr/share/java/mysql-connector-java.jar

      Y

      XAAUDIT.DB.IS_ENABLED Enable or disable database audit logging.

      Note: If this property is set to FALSE, Ranger will not store audit logs in the audit DB, and audit logs will not be visible in the Ranger UI. If you would like to access audit logs from the UI, set this value to TRUE.

      true

      Y

      XAAUDIT.DB.FLAVOUR Specifies the type of database used for audit logging (MYSQL,ORACLE)

      MYSQL

      Y

      XAAUDIT.DB.HOSTNAME Hostname of the audit database server

      localhost

      Y

      XAAUDIT.DB.DATABASE_NAME Audit database name

      ranger_audit

      Y

      XAAUDIT.DB.USER_NAME Username used for performing audit log inserts (should be same username used in the ranger-admin installation process)

      rangerlogger

      Y

      XAAUDIT.DB.PASSWORD database password associated with the above database user - for db audit logging

      rangerlogger

      Y

      HDFS Audit

       

       

      XAAUDIT.HDFS.IS_ENABLED Flag to enable/disable hdfs audit logging. If the hdfs audit logging is turned off, it will not log any access control to hdfs.

       

      Y

      XAAUDIT.HDFS.DESTINATION _DIRECTORY HDFS directory where the audit log will be stored

      hdfs://namenode.mycompany.com:8020/ranger/audit/%app-type%/%time:yyyyMMdd%

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _DIRECTORY Local directory where the audit log will be saved for intermediate storage

      /var/tmp/%app-type%/audit

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _DIRECTORY Local directory where the audit log will be archived after it is moved to hdfs

      /var/tmp/%app-type%/audit/archive

      Y

      XAAUDIT.HDFS.DESTINATION_FILE hdfs audit file name (format)

      %hostname%-audit.log

      Y

      XAAUDIT.HDFS.DESTINATION _FLUSH_INTERVAL_SECONDS hdfs audit log file writes are flushed to HDFS at regular flush interval

      900

      Y

      XAAUDIT.HDFS.DESTINATION _ROLLOVER_INTERVAL_SECONDS hdfs audit log file is rotated to write to a new file at a rollover interval specified here

      86400

      Y

      XAAUDIT.HDFS.DESTINATION _OPEN_RETRY_INTERVAL_SECONDS If hdfs audit log open() call is failed, it will be re-tried at this interval

      60

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FILE Local filename used to store in audit log (format)

      %time:yyyyMMdd-HHmm.ss%.log

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FLUSH_INTERVAL_SECONDS Local audit log file writes are flushed to filesystem at regular flush interval

      60

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _ROLLOVER_INTERVAL_SECONDS Local audit log file is rotated to write to a new file at a rollover interval specified here

      600

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _MAX_FILE_COUNT The maximum number of local audit log files will be kept in the archive directory

      10

      Y

      SSL (https connectivity to Policy Admin Tool)

       

       

      SSL_KEYSTORE_FILE_PATH Java Keystore Path where SSL key for the plug-in is stored. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      /etc/knox/conf/ranger-plugin-keystore.jks

      If SSL is enabled

      SSL_KEYSTORE_PASSWORD Password associated with SSL Keystore. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      myKeyFilePassword

      If SSL is enabled

      SSL_TRUSTSTORE_FILE_PATH Java Keystore Path where the trusted certificates are stored for verifying SSL connection to Policy Admin Tool. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      /etc/knox/conf/ranger-plugin-truststore.jks

      If SSL is enabled

      SSL_TRUSTSTORE_PASSWORD Password associated with Truststore file. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      changeit

      If SSL is enabled


  5. To enable the Knox plug-in, enter the following commands:

    cd /usr/hdp/<version>/ranger-knox-plugin
    ./enable-knox-plugin.sh
  6. Restart the Knox Gateway.

  7. To confirm that the Knox plug-in installation and configuration are complete, go to the Audit Tab of the Ranger Admin Console and check Plugins. You should see Knox listed there.

Installing the Ranger Storm Plug-in

The Ranger Storm plug-in integrates with Storm to enforce authorization policies.

This section describes how to perform the following administrative tasks: It assumes that Storm has already been installed, as described earlier in this guide.

  1. Create a Storm repository.

  2. Install the Storm plug-in and configure related Storm properties.

  3. Enable the Storm plug-in.

  4. Restart Storm.

Install the Storm Plug-in

  1. Create a Ranger HDFS policy using the instructions in Create an HDFS Policy in the Hadoop Security Guide.

    Make a note of the name you gave to this repository; you will need to use it again during HDFS plug-in setup.

  2. On the Nimbus server, install the Storm plug-in:

    1. Go to the home directory of the Storm plug-in:

      cd /usr/hdp/<version>/ranger-storm-plugin
    2. Edit the following Storm-related properties in the install.properties file:

      Table 13.9. Storm-Related Properties to Edit in the install.properties file

      Configuration Property Name

      Default/Example Value

      Mandatory?

      Policy Admin Tool

      POLICY_MGR_URL URL for policy admin

      http://policymanager.xasecure.net:6080

      Y

      REPOSITORY_NAME The repository name used in Policy Admin Tool for defining policies

      stormdev

      Y

      Audit Database

      SQL_CONNECTOR_JAR Path to SQL connector jar of the DB Flavor selected. The value should be the absolute path including the jar name.

      /usr/share/java/mysql-connector-java.jar (default)

      /usr/share/java/postgresql.jar

      /usr/share/java/sqljdbc4.jar

      /usr/share/java/ojdbc6.jar

      Y

      XAAUDIT.DB.IS_ENABLED Enable or disable database audit logging.

      Note: If this property is set to FALSE, Ranger will not store audit logs in the audit DB, and audit logs will not be visible in the Ranger UI. If you would like to access audit logs from the UI, set this value to TRUE.

      false (default) true

      Y

      XAAUDIT.DB.FLAVOUR Specifies the type of database used for audit logging (MYSQL,ORACLE, PostgreSQL 8.4.2, SQL Server 2012)

      MYSQL (default)

      Y

      XAAUDIT.DB.HOSTNAME Hostname of the audit database server

      localhost

      Y

      XAAUDIT.DB.DATABASE_NAME Audit database name

      ranger_audit

      Y

      XAAUDIT.DB.USER_NAME Username used for performing audit log inserts (should be same username used in the ranger-admin installation process)

      rangerlogger

      Y

      XAAUDIT.DB.PASSWORD Database password associated with the above database user - for db audit logging

      rangerlogger

      Y

      HDFS Audit

      XAAUDIT.HDFS.IS_ENABLED Flag to enable/disable hdfs audit logging. If the hdfs audit logging is turned off, it will not log any access control to hdfs.

      false

      Y

      XAAUDIT.HDFS.DESTINATION _DIRECTORY HDFS directory where the audit log will be stored

      hdfs://__REPLACE__NAME_NODE_HOST:8020/ranger/audit/%app-type%/%te:yyyyMMdd% (format) hdfs://namenode.mycompany.com:8020/ranger/audit/%app-type%/%time:yyyyMMdd%

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _DIRECTORY Local directory where the audit log will be saved for intermediate storage

      __REPLACE__LOG_DIR/%app-type%/audit (format) /var/log/%app-type%/audit

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _DIRECTORY Local directory where the audit log will be archived after it is moved to hdfs

      __REPLACE__LOG_DIR/%app-type%/audit/archive (format) /var/log/%app-type%/audit/archive

      Y

      XAAUDIT.HDFS.DESTINATION_FILE hdfs audit file name (format)

      %hostname%-audit.log (default)

      Y

      XAAUDIT.HDFS.DESTINATION _FLUSH_INTERVAL_SECONDS hdfs audit log file writes are flushed to HDFS at regular flush interval

      900 (default)

      Y

      XAAUDIT.HDFS.DESTINATION _ROLLOVER_INTERVAL_SECONDS hdfs audit log file is rotated to write to a new file at a rollover interval specified here

      86400 (default)

      Y

      XAAUDIT.HDFS.DESTINATION _OPEN_RETRY_INTERVAL_SECONDS If hdfs audit log open() call is failed, it will be re-tried at this interval

      60 (default)

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FILE Local filename used to store in audit log (format)

      %time:yyyyMMdd-HHmm.ss%.log (default)

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _FLUSH_INTERVAL_SECONDS Local audit log file writes are flushed to filesystem at regular flush interval

      60 (default)

      Y

      XAAUDIT.HDFS.LOCAL_BUFFER _ROLLOVER_INTERVAL_SECONDS Local audit log file is rotated to write to a new file at a rollover interval specified here

      600 (default)

      Y

      XAAUDIT.HDFS.LOCAL_ARCHIVE _MAX_FILE_COUNT The maximum number of local audit log files will be kept in the archive directory

      10 (default)

      Y

      SSL Information (https connectivity to policy Admin Tool)

      SSL_KEYSTORE_FILE_PATH Java Keystore Path where SSL key for the plug-in is stored. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      /etc/storm/conf/ranger-plugin-keystore.jks (default)

      If SSL is enabled

      SSL_KEYSTORE_PASSWORD Password associated with SSL Keystore. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      myKeyFilePassword (default)

      If SSL is enabled

      SSL_TRUSTSTORE_FILE_PATH Java Keystore Path where the trusted certificates are stored for verifying SSL connection to Policy Admin Tool. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      /etc/storm/conf/ranger-plugin-truststore.jks (default)

      If SSL is enabled

      SSL_TRUSTSTORE_PASSWORD Password associated with Truststore file. Is used only if SSL is enabled between Policy Admin Tool and Plugin; If SSL is not Enabled, leave the default value as it is - do not set as EMPTY if SSL not used.

      changeit (default)

      If SSL is enabled


  3. Enable the Storm plug-in by entering the following commands:

    cd /usr/hdp/<version>/ranger-storm-plugin
    ./enable-storm-plugin.sh
  4. Restart Storm.

  5. To confirm that the Storm plug-in installation and configuration are complete, go to the Audit Tab of the Ranger Admin Console and check Plugins. You should see Storm listed there.