Also available as:
loading table of contents...

Setting up PCAP to View Your Raw Data

The PCAP data source creates a Storm topology that can rapidly ingest raw data directly into HDFS from Kafka. As a result, you can store all of your cybersecurity data in its raw form in HDFS and review or query it at a later date. HCP supports two PCAP components:

  • The pycapa tool aimed at low-volume packet capture

    Pycapa is a open-source Python-based probe created by Cisco.

  • The Fastcapa tool aimed at high-volume packet capture.

    Fastcapa is a probe that performs fast network packet capture by leveraging Linux kernel-bypass and user space networking technology. The probe will bind to a network interface, capture network packets, and send the raw packet data to Kafka. This provides a scalable mechanism for ingesting high-volumes of network packet data into a Hadoop cluster.

    Fastcapa leverages the Data Plane Development Kit (DPDK). DPDK is a set of libraries and drivers to perform fast packet processing in Linux user space.

The rest of this chapter provides or points to instructions for setting up pycapa and Fastcapa and using PCAP and Fastcapa: