Using Apache Storm
Also available as:
PDF

Configuring KafkaSpout to Connect to a Secure Kafka Cluster

To connect to a Kerberized Kafka topic:

  1. Code: Add spoutConfig.securityProtocol=PLAINTEXTSASL to your Kafka Spout configuration.

  2. Configuration: Add a KafkaClient section (excerpted from /usr/hdp/current/kafka-broker/config/kafka_jaas.conf) to /usr/hdp/current/storm-supervisor/conf/storm_jaas.conf:

      KafkaClient
      { com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true renewTicket=true serviceName="kafka"; }
      ;
      ...
  3. Setup: Add a Kafka ACL for the topic:

    /usr/hdp/current/kafka-broker/bin/kafka-acls.sh --topic TEST --add allowhosts * --allowprincipals "User:stormusr" --operations DESCRIBE,READ --config /usr/hdp/current/kafka-broker/config/server.properties