Installing HDP Manually
Also available as:
PDF
loading table of contents...

(Optional) Configure Kerberos Authentication for Storm

Storm supports authentication using several models. This topic describes how to configure your Storm installation to use Kerberos authentication. At a high level, administrators must perform the tasks in this section.

Create Keytabs and Principals for Storm Daemons

Storm requires a principal and keytab when using Kerberos for authentication. A principal name in a given realm consists of a primary name and an instance name, the FQDN of the host that runs the service, in this case Storm. As services do not log in with a password to acquire their tickets, the authentication credentials for the service principal are stored in a keytab file, which is extracted from the Kerberos database and stored locally with the service principal on the service component host. First, create the principal using mandatory naming conventions. Then, create the keytab file with information from the new principal and copy the keytab to the keytab directory on the appropriate Storm host.

[Note]Note

Principals can be created either on the Kerberos Key Distribution Center (KDC) host or over the network using an “admin” principal. The following instructions assume you are using the KDC machine and using the kadmin.local command line administration utility. Using kadmin.local on the KDC machine allows you to create principals without needing to create a separate "admin" principal before you start.

Perform the following procedure on the host that runs KDC.

  1. Make sure that you have performed the steps in Securing ZooKeeper with Kerberos.

  2. Create a principal for the Nimbus server and the Storm DRPC daemon:

    sudo kadmin.local -q 'addprinc storm/<STORM_HOSTNAME>@STORM.EXAMPLE.COM'

  3. Create a keytab for the Nimbus server and the Storm DRPC daemon:

    sudo kadmin.local -q "ktadd -k /tmp/storm.keytab storm/<STORM_HOSTNAME>@STORM.EXAMPLE.COM"

  4. Copy the keytab to the Nimbus node and the node that runs the Storm DRPC daemon.

  5. Run the following command to create a principal for the Storm UI daemon, the Storm Logviewer daemon, and the nodes running the process controller, such as Supervisor. A process controller is used to start and stop the Storm daemons.

    sudo kadmin.local -q 'addprinc storm@STORM.EXAMPLE.COM'

  6. Create a keytab for the Storm UI daemon, the Storm Logviewer daemon, and Supervisor:

    sudo kadmin.local -q "ktadd -k /tmp/storm.keytab storm@STORM.EXAMPLE.COM"

  7. Copy the keytab to the cluster nodes running the Storm UI daemon, the Storm Logviewer daemon, and Supervisor.

Update the jaas.conf Configuration File

Both Storm and ZooKeeper use Java Authentication and Authorization Services (JAAS), an implementation of the Pluggable Authentication Model (PAM), to authenticate users. Administrators must update the jaas.conf configuration file with the keytab and principal information from the last step. The file must appear on all Storm nodes, the Nimbus node, the Storm DRPC node, and all Gateway nodes. However, different cluster nodes require different stanzas, as indicated in the following table:

Table 18.1. Required jaas.conf Sections for Cluster Nodes

Cluster Node

Required Sections in jaas.conf

Storm

StormClient

Nimbus

StormServer, Client

DRPC

StormServer

Supervisor

StormClient, Client

Gateway

StormClient (different structure than used on Storm and Supervisor nodes)

ZooKeeper

Server


[Note]Note

JAAS ignores unnecessary sections in jaas.conf. Administrators can put all sections in all copies of the file to simplify the process of updating it. However, the StormClient stanza for the Gateway nodes uses a different structure than the StormClient stanza on other cluster nodes. In addition, the StormServer stanza for the Nimbus node requires additional lines, as does the zoo.cfg configuration file for the ZooKeeper nodes.

The following example jaas.conf file contains all sections and includes information about the keytabs and principals generated in the previous step.

StormServer {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/keytabs/storm.keytab"
storeKey=true
useTicketCache=false
principal="storm/storm.example.com@STORM.EXAMPLE.COM";
};

StormClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/keytabs/storm.keytab"
storeKey=true
useTicketCache=false
serviceName="storm"
principal="storm@STORM.EXAMPLE.COM";
};

Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/keytabs/storm.keytab"
storeKey=true
useTicketCache=false
serviceName="zookeeper"
principal="storm@STORM.EXAMPLE.COM";
};

The StormServer section for the Nimbus node must have the following additional lines:

StormServer {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/keytabs/storm.keytab"
storeKey=true
useTicketCache=false
principal="storm/storm.example.com@STORM.EXAMPLE.COM";
};

The StormClient stanza for the Gateway nodes must have the following structure.

StormClient {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=false
useTicketCache=true
serviceName="$nimbus_user";
};

The Server stanza for the ZooKeeper nodes must have the following structure:

Server {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/keytabs/zk.keytab"
storeKey=true
useTicketCache=false
serviceName="zookeeper"
principal="zookeeper/zk1.example.com@STORM.EXAMPLE.COM";
};

In addition, add the following childopts lines to the stanzas for the nimbus, ui, and supervisor:

nimbus.childopts: "-Xmx1024m -Djava.security.auth.login.config=/path/to/jaas.conf"
ui.childopts: "-Xmx768m -Djava.security.auth.login.config=/path/to/jaas.conf"
supervisor.childopts: "-Xmx256m -Djava.security.auth.login.config=/path/to/jaas.conf"
[Note]Note

When starting ZooKeeper, include the following command-line option so that ZooKeeper can find jaas.conf:

-Djava.security.auth.login.config=/jaas/zk_jaas.conf

Update the storm.yaml Configuration File

To enable authentication with Kerberos, add the following lines to the storm.yaml configuration file:

storm.thrift.transport: "backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin"
java.security.auth.login.config: "/path/to/jaas.conf"
nimbus.authorizer: "backtype.storm.security.auth.authorizer.SimpleACLAuthorizer" 
storm.principal.tolocal: "backtype.storm.security.auth.KerberosPrincipalToLocal" 
storm.zookeeper.superACL: "sasl:storm"
nimbus.admins: - "storm" 
nimbus.supervisor.users: - "storm" 
nimbus.childopts: "-Xmx1024m -Djavax.net.debug=ssl -Dsun.security.krb5.debug=true -Djava.security.auth.login.config=/vagrant/storm_jaas.conf -Djava.security.krb5.realm=EXAMPLE.COM -Djava.security.krb5.kdc=kdc.example.com" 
ui.childopts: "-Xmx768m -Djavax.net.debug=ssl -Dsun.security.krb5.debug=true -Djava.security.auth.login.config=/vagrant/storm_jaas.conf -Djava.security.krb5.realm=EXAMPLE.COM -Djava.security.krb5.kdc=kdc.example.com" 
supervisor.childopts: "-Xmx256m -Djavax.net.debug=ssl -Dsun.security.krb5.debug=true -Djava.security.auth.login.config=/vagrant/storm_jaas.conf -Djava.security.krb5.realm=EXAMPLE.COM -Djava.security.krb5.kdc=example.host1.com" 
ui.filter: "org.apache.hadoop.security.authentication.server.AuthenticationFilter" 
ui.filter.params: "type": "kerberos""kerberos.principal": "HTTP/nimbus.example.com""kerberos.keytab": "/vagrant/keytabs/http.keytab""kerberos.name.rules": "RULE:[2:$1@$0]([jt]t@.*EXAMPLE.COM)s/.*/$MAPRED_USER/ RULE:[2:$1@$0]([nd]n@.*EXAMPLE.COM)s/.*/$HDFS_USER/DEFAULT"