Chapter 2. Configuring Ports

The tables below specify which ports must be opened for which ecosystem components to communicate with each other. Make sure the appropriate ports are opened before you install HDP.

Accumulo Ports

The following table lists the default ports used by the various Accumulo services. (Note: Neither of these services are used in a standard HDP installation.)

Table 2.1. Accumulo Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters
Master Master nodes (Active master and any standby) 9999 The Master thrift server Yes (client API needs) master.port.client in accumulo-site.xml
TabletServer Slave nodes 9997 The TabletServer thrift server Yes (client API needs) tserver.port.client in accumulo-site.xml
Garbage Collector GC nodes (Active GC and any standby) 50091 The GarbageCollector thrift server No gc.port.client in accumulo-site.xml
Monitor Monitor nodes (Active Monitor and any standby) 50095 HTTP(S) Metrics/Monitoring of an Accumulo instance Yes monitor.port.client in accumulo-site.xml
Monitor log aggregation Monitor nodes (Active Monitor and any standby) 4560 Log4j socket which accepts logs forwarded from other Accumulo services No monitor.port.log4j in accumulo-site.xml
Tracer Tracer nodes 12234 The Tracer thrift server Yes (if enabled) trace.port.client in accumulo-site.xml
Thrift Proxy (optional) Proxy nodes 42424 The Thrift Proxy server Yes (if enabled) port in proxy.properties
TabletServer Replication Service Slave nodes 10002 TabletServer Thrift service supporting multi-instance Accumulo replication No replication.receipt.service.port in accumulo-site.xml
Master Replication Service Master nodes (Active master and any standby) 10001 Master Thrift service supporting multi-instance Accumulo replication No master.replication.coordinator.port in accumulo-site.xml

Flume Ports

The following table lists the default ports used by the various Flume services. (Note: Neither of these services are used in a standard HDP installation.)

Table 2.2. Flume Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters
Flume Flume Agent 41414 TCP Flume performance metrics in JSON format Yes (client API needs) master.port.client in accumulo-site.xml
Flume HDFS Sink 8020 TCP Communication from Flume into the Hadoop cluster's NameNode Yes (client API needs) tserver.port.client in accumulo-site.xml
Flume HDFS Sink 9000 TCP Communication from Flume into the Hadoop cluster's NameNode No gc.port.client in accumulo-site.xml
Flume HDFS Sink 50010 TCP Communication from Flume into the Hadoop cluster's HDFS DataNode No
Flume HDFS Sink 50020 TCP Communication from Flume into the Hadoop cluster's HDFS DataNode No
Flume HBase Sink 2181 TCP Communication from Flume into the Hadoop cluster's Zookeeper No
Flume HBase Sink 60020 TCP Communication from Flume into the Hadoop cluster's HBase Regionserver No
Flume All Other Sources and Sinks Variable Variable Ports and protocols used by Flume sources and sinks No Refer to the flume configuration file(s) for ports actually in use. Ports in use are specificed using the port keyword in the Flume configuration file. By default Flume configuration files are located in /etc/flume/conf on Linux and c:\hdp\flume-1.4.0.x.y.z\conf on Windows

HDFS Ports

The following table lists the default ports used by the various HDFS services.

Table 2.3. HDFS Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

NameNode WebUI

Master Nodes (NameNode and any back-up NameNodes) 50070 http Web UI to look at current status of HDFS, explore file system Yes (Typically admins, Dev/Support teams) dfs.http.address
50470 https Secure http service dfs.https.address

NameNode metadata service

8020/9000 IPC

File system metadata operations

Yes (All clients who directly need to interact with the HDFS) Embedded in URI specified by fs.default.name

DataNode

All Slave Nodes

50075

http

DataNode WebUI to access the status, logs etc.

Yes (Typically admins, Dev/Support teams) dfs.datanode.http.address

50475

https

Secure http service

dfs.datanode.https.address

50010

Data transfer

dfs.datanode.address

50020

IPC

Metadata operations

No dfs.datanode.ipc.address
Secondary NameNode Secondary NameNode and any backup Secondanry NameNode

50090

http

Checkpoint for NameNode metadata

No dfs.secondary.http.address

MapReduce Ports: The following table lists the default ports used by the various MapReduce services.

Table 2.4. MapReduce Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

MapReduce

10020 http MapReduce JobHistory server address mapreduce.jobhistory.address

MapReduce

19888 http MapReduce JobHistory webapp address mapreduce.jobhistory.webapp.address
MapReduce 13562 http MapReduce Shuffle Port mapreduce.shuffle.port

YARN Ports: The following table lists the default ports used by the various YARN services.

Table 2.5. YARN Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

Resource Manager WebUI

Master Nodes (Resource Manager and any back-up Resource Manager node) 8088 http Web UI for Resource Manager Yes yarn.resourcemanager.webapp.address

Resource Manager

Master Nodes (Resource Manager Node) 8050 IPC

For application submissions

Yes (All clients who need to submit the YARN applications including Hive, Hive server, Pig) Embedded in URI specified by yarn.resourcemanager.address
Resource Manager Master Nodes (Resource Manager Node) 8025 http For application submissions Yes (All clients who need to submit the YARN applications including Hive, Hive server, Pig) yarn.resourcemanager.resource-tracker.address

Scheduler

Master Nodes (Resource Manager Node)

8030

http Scheduler Address Yes (Typically admins, Dev/Support teams) yarn.resourcemanager.scheduler.address

Resource Manager

Master Nodes (Resource Manager Node)

8141

http Scheduler Address Yes (Typically admins, Dev/Support teams) yarn.resourcemanager.admin.address

NodeManager

Master Nodes (NodeManager) and Slave Nodes

45454

http NodeManager Address Yes (Typically admins, Dev/Support teams) yarn.nodemanager.address

Timeline Server

Master Nodes

10200

http Timeline Server Address Yes (Typically admins, Dev/Support teams) yarn.timeline-service.address

Timeline Server

Master Nodes

8188

http Timeline Server Webapp Address Yes (Typically admins, Dev/Support teams) yarn.timeline-service.webapp.address

Timeline Server

Master Nodes

8190

https Timeline Server Webapp https Address Yes (Typically admins, Dev/Support teams) yarn.timeline-service.webapp.https.address

Job History Service

Master Nodes

19888

https Job History Service Yes (Typically admins, Dev/Support teams) yarn.log.server.url

Hive Ports

The following table lists the default ports used by the various Hive services. (Note: Neither of these services are used in a standard HDP installation.)

Table 2.6. Hive Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

Hive Server

Hive Server machine (Usually a utility machine) 10000 Service for programatically (Thrift/JDBC) connecting to Hive Yes (Clients who need to connect to Hive either programatically or through UI SQL tools that use JDBC) ENV Variable HIVE_PORT

Hive Web UI

Hive Server machine (Usually a utility machine) 9999 http

Web UI to explore Hive schemas

Yes hive.hwi.listen.port

Hive Metastore

9933 http Yes (Clients that run Hive, Pig and potentially M/R jobs that use HCatalog) hive.metastore.uris

HBase Ports

The following table lists the default ports used by the various HBase services.

Table 2.7. HBase Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

HMaster

Master Nodes (HBase Master Node and any back-up HBase Master node) 60000 Yes hbase.master.port

HMaster Info Web UI

Master Nodes (HBase master Node and back up HBase Master node if any) 60010 http The port for the HBase­Master web UI. Set to -1 if you do not want the info server to run. Yes hbase.master.info.port

Region Server

All Slave Nodes 60020 Yes (Typically admins, dev/support teams) hbase.regionserver.port

Region Server

All Slave Nodes 60030 http Yes (Typically admins, dev/support teams) hbase.regionserver.info.port
HBase REST Server (optional) All REST Servers 8080 http The port used by HBase Rest Servers. REST servers are optional, and not installed by default Yes hbase.rest.port
HBase REST Server Web UI (optional) All REST Servers 8085 http The port used by HBase Rest Servers web UI. REST servers are optional, and not installed by default Yes (Typically admins, dev/support teams) hbase.rest.info.port
HBase Thrift Server (optional) All Thrift Servers 9090 The port used by HBase Thrift Servers. Thrift servers are optional, and not installed by default Yes
HBase Thrift Server Web UI (optional) All Thrift Servers 9095 The port used by HBase Thrift Servers web UI. Thrift servers are optional, and not installed by default Yes (Typically admins, dev/support teams) hbase.thrift.info.port

Oozie Ports: The following table lists the default ports used by Oozie.

Table 2.8. Oozie Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters
Oozie Oozie Server 11000 TCP The port Oozie server runs. Yes OOZIE_HTTP_PORT in oozie_env.sh
Oozie Oozie Server 11001 TCP The admin port Oozie server runs. No OOZIE_ADMIN_PORT in oozie_env.sh
Oozie Oozie Server 11443 TCP The port Oozie server runs when using HTTPS. Yes OOZIE_HTTPS_PORT in oozie_env.sh

Sqoop Ports: The following table lists the default ports used by Sqoop.

Table 2.9. Sqoop Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters
Sqoop Metastore 16000 TCP Connection between Sqoop and the metastore No sqoop.metastore.server.port
Sqoop JDBC Listener Varies, depends on target database. For example, if moving data from MySQL, TCP port 3306 must be open. TCP Outbound port from the Hadoop cluster to the database. Varies depending on Database No

Storm Ports

The following table lists the default ports used by Storm.

Table 2.10. Storm Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

Zookeeper Port

2181 Port used by localhost to talk to ZooKeeper. storm.zookeeper.port
DRPC Port 3772 drpc.port

DRPC Invocations Port

3773 drpc.invocations.port
Nimbus Thrift Port

6627

nimbus.thrift.port
Supervisor Slots Ports 6700, 6701, 6702, 7603 Defines the amount of workers that can be run on this machine. Each worker is assigned a port to use for communication. supervisor.slots.ports
Logviewer Port 8000 logviewer.port
UI Port 8080 ui.port
Ambari Port 8744 ambari.port

ZooKeeper Ports

Table 2.11. ZooKeeper Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters
ZooKeeper Server All ZooKeeper Nodes 2888 Port used by ZooKeeper peers to talk to each other. See here for more information. No hbase.zookeeper.peerport
ZooKeeper Server All ZooKeeper Nodes 3888 Port used by ZooKeeper peers to talk to each other.See here for more information. No hbase.zookeeper.leaderport
ZooKeeper Server All ZooKeeper Nodes 2181 Property from ZooKeeper's config zoo.cfg. The port at which the clients will connect. No hbase.zookeeper.property.clientPort

MySQL Ports: The following table lists the default ports used by the various MySQL services.

Table 2.12. MySQL Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters
MySQL MySQL database server 3306