Configuring Authentication with Kerberos
Also available as:
PDF
loading table of contents...

Create Kafka Topics

When you use a script, command, or API to create a topic, an entry is created under ZooKeeper. The only user with access to ZooKeeper is the service account running Kafka (by default, kafka). Therefore, the first step toward creating a Kafka topic on a secure cluster is to run kinit, specifying the Kafka service keytab. The second step is to create the topic.

  1. Run kinit, specifying the Kafka service keytab. For example:
    kinit -k -t /etc/security/keytabs/kafka.service.keytab kafka/c6401.ambari.apache.org@EXAMPLE.COM
  2. Next, create the topic. Run the kafka-topics.sh command-line tool with the following options:

    /bin/kafka-topics.sh --zookeeper <hostname>:<port> --create --topic <topic-name> --partitions <number-of-partitions> --replication-factor <number-of-replicating-servers>

    For more information about kafka-topics.sh parameters, see Basic Kafka Operations on the Apache Kafka website.

    /bin/kafka-topics.sh --zookeeper c6401.ambari.apache.org:2181 --create --topic test_topic --partitions 2 --replication-factor 2
    
    Created topic "test_topic".
  3. Add permissions:

    By default, permissions are set so that only the Kafka service user has access; no other user can read or write to the new topic. In other words, if your Kafka server is running with principal $KAFKA-USER, only that principal will be able to write to ZooKeeper.

    For information about adding permissions, see “Authorizing Access when Kerberos is Enabled”.