Individual Sensor Enrichments

HCP includes the following individual sensor enrichments:

Geo: Provides GeoIP information, which includes coordinates, city, state, and country information, to any external IP address.
Asset: Provides the host name for an IP address. If the IP address is known, then the enrichment provides everything else that is known of the asset from the LDAP, AD, or enterprise inventory stores.
User: Provides the user that owns the session/alert associated with the IP-application pair.

The JSON documents for the individual enrichment configurations are structured as follows:

Table 5.1. Individual Enrichment Configuration Fields

Field	Description	Example
`fieldToTypeMap`	In the case of a simple HBase enrichment (in other words, a key/value lookup), the mapping between fields and the enrichment types associated with those fields must be known. This enrichment type is used as part of the HBase key.	"fieldToTypeMap" : { "ip_src_addr" : [ "asset_enrichment" ] }
`fieldMap`	The map of enrichment bolts names to configuration handlers which know how to split the message up. The simplest of which is just a list of fields. More complex examples would be the STELLAR enrichment which provides STELLAR statements. Each field is sent to the enrichment referenced in the key.	"fieldMap": {"hbaseEnrichment": ["ip_src_addr","ip_dst_addr"]}
`config`	The general configuration for the enrichment.	"config": {"typeToColumnFamily": { "asset_enrichment" : "cf" } }

The config map is intended to house enrichment-specific configurations. For example, hbaseEnrichment specifies the mappings between the enrichment types to the column families.

The fieldMapcontents contain the routing and configuration information for the enrichments. Routing defines how the messages is split up and sent to the enrichment adapter bolts. The simplest fieldMapcontents provides a simple list as in:

 "fieldMap": {
      "geo": [
        "ip_src_addr",
        "ip_dst_addr"
      ],
      "host": [
        "ip_src_addr",
        "ip_dst_addr"
      ],
      "hbaseEnrichment": [
        "ip_src_addr",
        "ip_dst_addr"
      ]
      }

For the geo, host, and hbaseEnrichment, this is sufficient. However, more complex enrichments might contain their own configuration. Currently, the stellar enrichment requires a more complex configuration, such as:

"fieldMap": {
       ...
      "stellar" : {
        "config" : {
          "numeric" : {
                      "foo": "1 + 1"
                      }
          ,"ALL_CAPS" : "TO_UPPER(source.type)"
        }
      }
    }

Whereas the simpler enrichments just need a set of fields explicitly stated so they can be separated from the message and sent to the enrichment adapter bolt for enrichment and ultimately joined back in the join bolt, the stellar enrichment has its set of required fields implicitly stated through usage. For instance, if your Stellar statement references a field, it should be included and if not, then it should not be included. We did not want to require users to make explicit the implicit.

The other way in which the Stellar enrichment is somewhat more complex is in how the statements are executed. In the general purpose case for a list of fields, those fields are used to create a message to send to the enrichment adapter bolt and that bolt's worker will handle the fields one by one in serial for a given message. For Stellar enrichment, we wanted to have a more complex design so that users could specify the groups of stellar statements sent to the same worker in the same message (and thus executed sequentially). Consider the following configuration:

    "fieldMap": {
      "stellar" : {
        "config" : {
          "numeric" : {
                      "foo": "1 + 1"
                      "bar" : TO_LOWER(source.type)"
                      }
         ,"text" : {
                   "ALL_CAPS" : "TO_UPPER(source.type)"
                   }
        }
      }
    }

We have a group called numeric whose Stellar statements will be executed sequentially. In parallel to that, we have the group of Stellar statements under the group text executing. The intent here is to allow you to not force higher latency operations to be done sequentially.

​Individual Sensor Enrichments

Individual Sensor Enrichments