Create an Index Template

To work with a new data source data in the Metron dashboard, you need to ensure that the data is landing in the search index (Elasticsearch) with the correct data types. You can achieve this by defining an index template.

1. Run the following command to create an index template for the new data source.

   curl -XPOST $SEARCH_HOST:$SEARCH_PORT/_template/$DATASOURCE_index -d '
   {
     "template": "sensor1_index*",
     "mappings": {
       "sensor1_doc": {
         "properties": {
           "timestamp": {
             "type": "date",
             "format": "epoch_millis"
           },
           "ip_src_addr": {
             "type": "ip"
           },
           "ip_src_port": {
             "type": "integer"
           },
           "ip_dst_addr": {
             "type": "ip"
           },
           "ip_dst_port": {
             "type": "integer"
           }
         }
       }
     }
   }'

   The code sample defines an index template for a new sensor called 'sensor1'.
   The example assumes the following:
   • The template applies to any indices that are named sensor1_index*.
   • The index has one document type that must be named sensor1_doc.
   • The index contains timestamps.
   • The properties section defines the types of each field. This example defines the five common fields that most sensors contain.
   • Additional fields can be added following the five that are already defined.
   By default, Elasticsearch will attempt to analyze all fields of type string. This means that Elasticsearch will tokenize the string and perform additional processing to enable free-form text search. In many cases, you want to treat each of the string fields as enumerations. This is why most fields in the index template are `not_analyzed`.
2. An index template will only apply for indices that are created after the template is created. Delete the existing indices for the new data source so that new ones can be generated with the index template.

   curl -XDELETE $SEARCH_HOST:9200/$DATSOURCE*

3. Wait for the new data source index to be re-created.
   This might take a minute or two based on how fast the new data source data is being consumed in your environment.

   curl -XGET $SEARCH_HOST:9200/$DATASOURCE*