Adding a New Telemetry Source
Also available as:

Creating Parsers

Parsers transform raw data into JSON messages suitable for downstream enrichment and indexing by HCP. There is one parser for each data source and HCP pipes the information to the Enrichment/Threat Intelligence topology.

You can transform the field output in the JSON messages into information and formats that make the output more useful. For example, you can change the timestamp field output from GMT to your timezone.

You must make two decisions before you parse a new data source:

  • Type of parser to use

    HCP supports three types of parsers:

    HCP features several built-in parsers that support many common security devices.
    General Purpose

    HCP supports three general purpose parsers: Grok, CSV, and JSON map.

    • Grok - Regular expression-based parser extracts HCP values; ideal for ingesting structured or semi-structured logs that are well understood and telemetries with lower volumes of traffic
    • CSV - Maps CSV columns to HCP events
    • JSON Map - Maps JSON documents into HCP events

    A Java parser is appropriate for a telemetry type that is complex to parse, with high volumes of traffic.

  • How to parse

    HCP enables you to parse a new data source and transform data fields using the HCP Management module or the command line interface