3.6. Configure ports

HDP uses multiple ports for communication with clients and between service components. To enable HDP communication, open the specific ports that HDP uses.

To open specific ports only, you can set the access rules in Windows.

For example, the following command will open up port 80 in the active Windows Firewall:

netsh advfirewall firewall add rule name=AllowRPCCommunication dir=in action=allow protocol=TCP localport=80

For example, the following command will open up ports 49152-65535 in the active Windows Firewall:

netsh advfirewall firewall add rule name=AllowRPCCommunication dir=in action=allow protocol=TCP localport=49152-65535

The tables below specify which ports must be opened for which ecosystem components to communicate with each other.

Make sure that appropriate ports are opened before you install HDP.

HDFS Ports: The following table lists the default ports used by the various HDFS services.

Table 1.2. HDFS Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

NameNode WebUI

Master Nodes (NameNode and any back-up NameNodes) 50070 http Web UI to look at current status of HDFS, explore file system Yes (Typically admins, Dev/Support teams) dfs.http.address

NameNode metadata service

8020/9000 IPC

File system metadata operations

Yes (All clients who directly need to interact with the HDFS) Embedded in URI specified by fs.default.name

DataNode

All Slave Nodes

50075

http

DataNode WebUI to access the status, logs etc.

Yes (Typically admins, Dev/Support teams) dfs.datanode.http.address

50010

Data transfer

dfs.datanode.address

50020

IPC

Metadata operations

No dfs.datanode.ipc.address
Secondary NameNode Secondary NameNode and any backup Secondanry NameNode

50090

http

Checkpoint for NameNode metadata

No dfs.secondary.http.address

YARN Ports: The following table lists the default ports used by the various YARN services.

Table 1.3. YARN Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

Resource Manager WebUI

Master Nodes (Resource Manager and any back-up Resource Manager node) 8088 http Web UI for Resource Manager Yes yarn.resourcemanager.webapp.address
Master Nodes (Resource Manager and any back-up Resource Manager node) 8090 https Web UI for Resource Manager Yes yarn.resourcemanager.webapp.https.address

Resource Manager

Master Nodes (Resource Manager Node) 8032 IPC

For application submissions

Yes (All clients who need to submit the YARN applications including Hive, Hive server, Pig) Embedded in URI specified by yarn.resourcemanager.address
Resource Manager Admin Interface Master Nodes (Resource Manager and any back-up Resource Manager node) 8033 Administrative interface Yes (Typically admins and support teams) yarn.resourcemanager.admin.address
Resource Manager Scheduler Master Nodes (Resource Manager and any back-up Resource Manager node) 8031 http Resource Manager Interface Yes (Typically admins, Dev/Support teams) yarn.resourcemanager.scheduler.address

NodeManager Web UI

All Slave Nodes

50060

http Yes (Typically admins, Dev/Support teams) yarn.nodemanager.webapp.address

Hive Ports: The following table lists the default ports used by the Hive services.

Table 1.4. Hive Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

HiveServer2

HiveServer2 machine (Usually a utility machine) 10001 thrift Service for programmatically (Thrift/JDBC) connecting to Hive Yes ENV Variable HIVE_PORT

Hive Server

Hive Server machine (Usually a utility machine) 10000 thrift Service for programmatically (Thrift/JDBC) connecting to Hive Yes (Clients who need to connect to Hive either programatically or through UI SQL tools that use JDBC) ENV Variable HIVE_PORT

Hive Metastore

9083 thrift Service for programmatically (Thrift/JDBC) connecting to Hive metadata Yes (Clients that run Hive, Pig and potentially M/R jobs that use HCatalog) hive.metastore.uris

WebHcat Port: The following table lists the default port used by the WebHCat service.

Table 1.5. WebHCat Port
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

WebHCat Server

Any utility machine 50111 http Web API on top of HCatalog and other Hadoop services Yes templeton.port

Table 1.6. HBase Ports
Service Servers Default Ports Used Protocol Description Need End User Access? Configuration Parameters

HMaster

Master Nodes (HBase Master Node and any back-up HBase Master node) 60000 Yes hbase.master.port

HMaster Info Web UI

Master Nodes (HBase master Node and back up HBase Master node if any) 60010 http The port for the HBase­Master web UI. Set to -1 if you do not want the info server to run. Yes hbase.master.info.port

Region Server

All Slave Nodes 60020 Yes (Typically admins, dev/support teams) hbase.regionserver.port

Region Server

All Slave Nodes 60030 http Yes (Typically admins, dev/support teams) hbase.regionserver.info.port
ZooKeeper All ZooKeeper Nodes 2888 Port used by ZooKeeper peers to talk to each other. See here for more information. No hbase.zookeeper.peerport
ZooKeeper All ZooKeeper Nodes 3888 Port used by ZooKeeper peers to talk to each other. See here for more information. hbase.zookeeper.leaderport
2181 Property from ZooKeeper's config zoo.cfg. The port at which the clients will connect. hbase.zookeeper.property.clientPort


loading table of contents...