Understanding Parsing a New Data Source to HCP
Parsers transform raw data into JSON messages suitable for downstream enrichment and indexing by HCP. There is one parser for each data source and HCP pipes the information to the Enrichment/Threat Intelligence topology.
You can transform the field output in the JSON messages into information and formats that make the output more useful. For example, you can change the timestamp field output from GMT to your timezone.
You must make two decisions before you parse a new data source:
Type of parser to use
HCP supports two types of parsers:
- General Purpose
HCP supports two general purpose parsers: Grok and CSV. These parsers are ideal for structured or semi-structured logs that are well understood and telemetries with lower volumes of traffic.
- Java
A Java parser is appropriate for a telemetry type that is complex to parse, with high volumes of traffic.
How to parse
HCP enables you to parse a new data source and transform data fields using the HCP Management module or the command line interface: