Using Apache Storm to Move Data
Also available as:
PDF

Configuring KafkaSpout for a Secure Kafka Cluster

To connect to a Kerberized Kafka topic:

  1. Code: Add spoutConfig.securityProtocol=PLAINTEXTSASL to your Kafka Spout configuration.
  2. Configuration: Add a KafkaClient section (excerpted from /usr/hdp/current/kafka-broker/config/kafka_jaas.conf) to /usr/hdp/current/storm-supervisor/conf/storm_jaas.conf:
    KafkaClient {
           com.sun.security.auth.module.Krb5LoginModule required
           useKeyTab=true
           keyTab="/etc/security/keytabs/stormusr.service.keytab"
           storeKey=true
           useTicketCache=false
           serviceName="kafka" 
           principal="stormusr/host.name@EXAMPLE.COM";
           };
  3. Setup: Add a Kafka ACL for the topic. For example:

    bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal user:stormusr --allow-hosts * --operations Read --topic TEST