Run Book - Technical Preview
Also available as:
PDF

Create an Index Template

To work with a new data source data in the Metron dashboard, you need to ensure that the data is landing in the search index (Elasticsearch) with the correct data types. You can achieve this by defining an index template.

[Note]Note

You will need to update the Index template after you add or change enrichments for a data source.

  1. Run the following command to create an index template for the Squid.

    curl -XPOST $SEARCH_HOST:$SEARCH_PORT/_template/squid_index -d '
    {
      "template": "squid_index*",
      "mappings": {
        "squid_doc": {
          "_timestamp": {
            "enabled": true
          },
          "properties": {
            "timestamp": {
              "type": "date",
              "format": "epoch_millis"
            },
            "ip_src_addr": {
              "type": "ip"
            },
            "ip_src_port": {
              "type": "integer"
            },
            "ip_dst_addr": {
              "type": "ip"
            },
            "ip_dst_port": {
              "type": "integer"
            }
          }
        }
      }
    }'

    The example assumes the following:

    • The template applies to any indices that are named squid_index*.

    • The index has one document type that must be named squid_doc.

    • The index contains timestamps.

    • The properties section defines the types of each field. This example defines the five common fields that most sensors contain.

    • Additional fields can be added following the five that are already defined.

    By default, Elasticsearch will attempt to analyze all fields of type string. This means that Elasticsearch will tokenize the string and perform additional processing to enable free-form text search. In many cases, and all cases for the Squid data, you want to treat each of the string fields as enumerations. This is why most fields in the index template are `not_analyzed`.

  2. [OPTIONAL] An index template will only apply for indices that are created after the template is created. If you want to delete existing indices for the new data source so that new ones can be generated with the index template, run the following command.

    [Important]Important

    Running this command will delete all existing indices.

    curl -XDELETE $SEARCH_HOST:9200/squid*
  3. Wait for the Squid index to be re-created.

    This might take a minute or two based on how fast the new data source data is being consumed in your environment.

    curl -XGET $SEARCH_HOST:9200/$DATASOURCE*