Security
Also available as:
PDF
loading table of contents...

Migrating Audit Logs from DB to Solr in Ambari Clusters

It is recommended that you store audits in both HDFS and Solr. Audit to DB is no longer recommended and the option is disabled in the Ambari UI. If your logs were previously stored on DB, you can migrate the logs to Solr.

[Note]Note

By default, Solr only indexes the last 30 days' logs.

Before you migrate your audit logs from DB to Solr, make sure your cluster meets the following requirements:

  • Solr must be installed and running (see Using Apache Solr for Ranger Audits).

  • All plug-ins must be upgraded and writing audit logs to Solr (i.e., plugins must not be writing audit logs to DB.)

  • The DB server must be running, and the credentials used to connect Audit to DB must be available.

  • Ranger must be running with the audit source as Solr, and the Solr URL must be configured.

To migrate your audit logs from DB to Solr, complete the following instructions:

  1. Configure the properties ranger.audit.source.type and ranger.audit.solr.urls:

    Property NameSample ValueLocation
    ranger.audit.source.typesolrRanger>Configs>Advanced>Advanced ranger-admin-site
    ranger.audit.solr.urlsSyntax: http://<solr_host>:<port>/solr/ranger_audits Example: http://192.168.0.2:8983/solr/ranger_auditsExample: http://192.168.0.2:8886/solr/ranger_auditsRanger>Configs>Ranger Audit
  2. Verify or enter the ranger.jpa.audit.jdbc.url value.

    After upgrading Ranger and changing the audit log destination from DB to Solr, Ambari may not automatically populate the required property values. If necessary, you can add these as custom properties from Ambari.

    1. Select Ranger>Configs>Advanced>Custom ranger-admin-site, then click Add Property....

    2. Enter the following information on the Add Property pop-up:

      • Type: preloaded with the value ranger-admin-site.xml

      • Key: enter ranger.jpa.audit.jdbc.url

      • Value: enter the JDBC audit string for your DB platform:

        Table 5.4. JDBC Audit String

        DB PlatformSyntaxExample Value
        MySQLjdbc:mysql://DB_HOST:PORT/audit_namejdbc:mysql://c6401.ambari.apache.org:3306/ranger_audit
        Oracle

        For Oracle SID:

        jdbc:oracle:thin:@AUDIT_HOST:PORT:SID

        jdbc:oracle:thin:@c6401.ambari.apache.org:1521:ORCL

        For Oracle Service Name:

        jdbc:oracle:thin:@//AUDIT_HOST[:PORT][/ServiceName]

        jdbc:oracle:thin:@//c6401.ambari.apache.org:1521/XE
        PostgreSQLjdbc:postgresql://AUDIT_HOST/audit_namejdbc:postgresql://c6401.ambari.apache.org:5432/ranger_audit
        MS SQLjdbc:sqlserver://AUDIT_HOST;databaseName=audit_namejdbc:sqlserver://c6401.ambari.apache.org:1433;databaseName=ranger_audit
        SQLAjdbc:sqlanywhere:host=AUDIT_HOST;database=audit_namejdbc:sqlanywhere:host=c6401.ambari.apache.org:2638;database=ranger_audit


  3. Restart Ranger Admin.

  4. Navigate to the Ranger admin directory and run the following command:

    $/path/to/java -Dlogdir=ews/logs -Dlog4j.configuration=db_patch.log4j.xml -cp ews/webapp/WEB-INF/classes/conf:ews/webapp/WEB-INF/classes/lib/*:ews/webapp/WEB-INF/:ews/webapp/META-INF/:ews/webapp/WEB-INF/lib/*:ews/webapp/WEB-INF/classes/:ews/webapp/WEB-INF/classes/META-INF:/usr/share/java/mysql-connector-java.jar org.apache.ranger.patch.cliutil.DbToSolrMigrationUtil

    If the script succeeds, it prints the following details on the screen:

    • Processing batch 'n' of total 'noOfBatches' (Where each batch contains 10000 rows.)

    • Total number of migrated audit logs.

    If the script fails to migrate data, it returns the error: Migration process failed, Please refer ranger_db_patch.log file.