Run Book - Technical Preview
Also available as:
PDF

Configuring the Profiler

The configuration for the Profiler topology is stored in ZooKeeper at /metron/topology/profiler. These properties also exist in the default installation of HCP at $METRON_HOME/config/zookeeper/profiler.json. You can change these values two ways: with Ambari or on disk and then uploaded to ZooKeeper using $METRON_HOME/bin/zk_load_configs.sh. The following task uses Ambari to configure the Profiler.

You might need to work with your Platform Engineer to modify Profiler values.

[Note]Note

The Profiler can persist any serializable object, not just numeric values.

To configure the Profiler, complete the following steps:

  1. Display the Ambari user interface and click the Services tab.

  2. Click Metron in the list of services, then click the Configs tab.

  3. Click the Profiler tab.

    Ambari displays a list of Profiler properties that you can use to configure Profiler.

    Figure 8.1. Ambari Profiler Properties


  4. Use these properties to configure the Profiler, then click the Save button near the top of the window.

The following table provides a list of the configurable properties and descriptions.

Table 8.1. Profiler Properties

Ambari Configs Field.Settings.Description
Kafka Input Topic Startprofiler_kafka_startOne of EARLIEST, LATEST, UNCOMMITTED_EARLIEST, UNCOMMITTED_LATEST
Profiler Setup Period Durationprofiler_period_durationThe duration of each profile period. This value should be defined along with profiler_period_duration_units.
Period Unitsprofiler_period_unitsThe units is used to specify the profiler_period_duration. This value should be define along with profiler_period_duration.
Time to Liveprofiler_ttlIf a message has not been applied to a Profile in this period of time, the Profile will be terminated and its resources will be cleaned up. This value should be defined along with profiler_ttl_units. This time-to-live does not affect the persisted Profile data in HBase. It only affects the state stored in memory during the execution of the latest profile period. This state will be deleted if the time_to_live is exceeded.
Time to Live Unitsprofiler_ttl_unitsThe units used to specify the profiler_ttl.
HBase Tableprofiler_hbase_tableThe name of the HBase table the profiler is written to. The profiler expects that the table exists and is writable.
HBase Table Column Familyprofiler_hbase_cfThe column family used to store profile data in HBase.
HBase Batch Sizeprofiler_hbase_batchThe number of puts that are written to HBase in a single batch.
HBase Flush Intervalprofiler_hbase_flush_intervalThe maximum number of seconds between batch writes to HBase.
Storm topology.worker.childoptsprofiler_topology_worker_childoptsExtra topology child opts for the storm opts.
Number of Workersprofiler_topology_workersThe profiler storm topology storm workers
Number of Acker Executorsprofiler_acker_executorsThe profiler storm topology acker executors

[Note]Note

The profiler input topic is bound to the enrichment output topic. If that enrichment output topic is changed, then the profiler will restart as well as the enrichment topology.

Figure 8.2. Enrichment Output Topic