Understanding Indexing

Currently, Hortonworks Cybersecurity Platform (HCP) supports the following indices:

Depending on how you configure the indexing topology, it can have HDFS and either Elasticsearch or Solr writers running.

The Indexing Configuration file is a JSON file stored in Apache ZooKeeper and on disk at $METRON_HOME/config/zookeeper/indexing.

Errors during indexing are sent to a Kafka queue called index_errors.

Within the sensor-specific configuration, you can configure the individual writers. The following parameters are currently supported:

index: The name of the index to write to (default is the name of the sensor).
batchSize: The size of the batch allowed to be written to the indices at once (defaulted is 1).
enabled: Whether the index or writer is enabled (default is true).