Hortonworks Cybersecurity Platform
Also available as:
loading table of contents...

Set up pycapa on a Kerberized Environment

The pycapa probe can be used in a Kerberized environment. However, setting up pycapa on a Kerberized environment requires different steps than setting up pycapa in an unkerberized environment.

  • Ensure you have installed Python 2.7
  • This installation assumes the following environment variables:
    • The Kafka broker is at kafka1:6667
    • Zookeeper is at zookeeper1:2181
    • The Kafka security protocol is SASL_PLAINTEXT
    • The keytab used is located at /etc/security/keytabs/metron.headless.keytab
    • The service principal is metron@EXAMPLE.COM
    • PYCAPA_HOME=/opt/pycapa
      PYTHON27_HOME =/opt/rh/python27/root
  1. Ensure that you have Simple Authentication and Security Layer (SASL) library libsasl or libsasl2 installed.
    On CentOS, you can be install the library with the following command:
    yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
  2. Build the Apache Kafka client library (librdkafka) with SASL support (--enable-sasl) and install it at your chosen $PREFIX:
    export PREFIX=/usr
    wget https://github.com/edenhill/librdkafka/archive/v0.11.5.tar.gz   -O - | tar -xz
    cd librdkafka-0.11.5/
    ./configure --prefix=$PREFIX
    make install
  3. Validate that librdkafka supports the SASL.
    Run the following command to ensure that SASL is returned as a built-in feature:
    $ examples/rdkafka_example -X builtin.features    
      builtin.features = gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins 
  4. If you have already installed confluent-kafka, remove the binary wheel python client before re-installing confluent-kafka.
    Repeat the command until the system says confluent-kafka it is no longer installed:
    pip uninstall -y confluent-kafka
  5. Install confluent-kafka:
    pip install --no-binary :all: confluent-kafka
  6. Grant access to your Kafka topic.
    In the following example the topic is simply named pcap.
    ${KAFKA_HOME}/bin/kafka-acls.sh \
      --authorizer kafka.security.auth.SimpleAclAuthorizer \
      --authorizer-properties zookeeper.connect=zookeeper1:2181 \
      --add \
      --allow-principal User:metron \
      --topic pcap
    ${KAFKA_HOME}/bin/kafka-acls.sh \
      --authorizer kafka.security.auth.SimpleAclAuthorizer \
      --authorizer-properties zookeeper.connect=zookeeper1:2181 \
      --add \
      --allow-principal User:metron \
      --group pycapa
  7. Use pycapa as you normally would, but append the following three additional parameters:
    • security.protocol
    • sasl.kerberos.keytab
    • sasl.kerberos.principal
    $ pycapa --producer \
            --interface eth0 \
            --kafka-broker kafka1:6667 \
            --kafka-topic pcap --max-packets 10 \
            -X security.protocol=SASL_PLAINTEXT \
            -X sasl.kerberos.keytab=/etc/security/keytabs/metron.headless  .keytab \
            -X sasl.kerberos.principal=metron-metron@METRONEXAMPLE.COM
        INFO:root:Connecting to Kafka; {'sasl.kerberos.principal':   'metron-metron@METRONEXAMPLE.COM', 'group.id': 'ORNLVWJZZUAA',   'security.protocol': 'SASL_PLAINTEXT', 'sasl.kerberos.keytab':   '/etc/security/keytabs/metron.headless.keytab', 'bootstrap.servers': 'kafka1:6667'}
        INFO:root:Starting packet capture
        INFO:root:Waiting for '1' message(s) to flush
        INFO:root:'10' packet(s) in, '10' packet(s) out