Creating Profiles
Also available as:

Streaming Profiler Properties

Use the profiler properties to configure the streaming profiler.

Table 1. Profiler Properties
Ambari Configs Field. Settings Description
Profiler Setup
Period Duration profiler.period.duration The duration of each profile period. This value should be defined along with profiler.period.duration.units.
Period Units profiler.period.units The units is used to specify the profiler.period.duration. This value should be define along with profiler.period.duration.
Window Duration profiler.window.duration The duration of each profile window.
Window Units profiler.window.duration.units The units used to specify the profiler.window.duration.
Time to Live profiler.ttl If a message has not been applied to a Profile in this period of time, the Profile will be terminated and its resources will be cleaned up. This value should be defined along with profiler.ttl.units. This time-to-live does not affect the persisted Profile data in HBase. It only affects the state stored in memory during the execution of the latest profile period. This state will be deleted if the is exceeded.
Time to Live Units profiler.ttl.units The units used to specify the profiler.ttl.
Window Time Lag profiler.window.log The maximum time lag for timestamps.
Window Lag Units profiler.window.log.units The units used to specify the profiler.window.lag.
Max Routes Per Bolt profiler.max.routes.per.bolt The maximum number of routes that will be maintained by the bolt. After this value is exceeded, lesser used routes will be evicted from the internal cache.
Kafka Writer Batch Size profiler.writer.batchSize The number of records to batch when writing to Kakfa.
Kafka Writer Batch Timeout profiler.writer.batchTimeout The timeout in ms for batching when writing to Kakfa.
topology.worker.childopts profiler.topology.worker.childopts Extra topology child opts for the storm opts.
Number of Workers profiler.topology.workers The profiler storm topology storm workers
Number of Acker Executors profiler.acker.executors The profiler storm topology acker executors
Profiler Topology Message Timeout topology.message.timeout.secs Maximum amount of time given to the topology to fully process a tuple tree from the core-storm API, or a batch from the Trident API, emitted by a spout. If the message is not acked within this time frame, Storm fails the operation on the spout. The default is 30 seconds.
Spout Max Pending Tuples topology.max.spout.pending Maximum number of messages that can be pending in a spout at any time. The default is null (no limit).
HBase Table profiler.hbase.table The name of the HBase table the profiler is written to. The profiler expects that the table exists and is writable.
HBase Table Column Family The column family used to store profile data in HBase.
HBase Batch Size profiler.hbase.batch The number of puts that are written to HBase in a single batch.
HBase Flush Interval profiler.hbase.flush.interval The maximum number of seconds between batch writes to HBase.
N/A profiler.writer.batchSize The size of the batch that is written to Kafka at once. Defaults to 15 (size of 1 disables batching).
N/A profiler.writer.batchTimeout The timeout after which a batch will be flushed even if batchSize has not been met. Optional. If unspecified or set to 0, it defaults to a system-determined duration which is a fraction of the Storm parameter topology.message.timeout.secs. Ignored if batchsize is 1 because this disables batching.