Also available as:

Chapter 6. SmartSense Ports and Traffic Flow

When deploying SmartSense in an enterprise environment, you must carefully plan your network architecture. SmartSense functionality relies on how multiple internal clusters create bundles and then send them through a central SmartSense Gateway to the hosted Hortonworks environment for analysis or to Hortonworks Support to troubleshoot support cases.

The following graphic illustrates how data traffic flows along various communication channels between cluster nodes and SmartSense ports:

The communication channels illustrated here are described in the following sections:

User Interface or Ambari View to HST Server

When using SmartSense without Ambari, users access the web UI directly, whereas when using Ambari, they use Ambari View to communicate with the server.

Source ComponentDestination ComponentDestination Port Purpose
User interface or Ambari ViewHST servertcp/9000 Web UI communication
Zeppelin Activity Explorer UIZeppelin servertcp/9060Web UI communication

HST Agent to HST Server

All communication between the HST server and HST agents is initiated by the agent, using the following ports:

Source ComponentDestination ComponentDestination PortTransport SecurityPurpose
HST AgentHST Servertcp/9440One-way SSLAgent registration
HST AgentHST Servertcp/9441Two-way SSLAnonymized bundle transfer

HST agents register themselves with the HST server, and when invoked to capture data, use the same port to securely transmit captured data back to the HST server.

HST Server to SmartSense Gateway

All communication between the HST server and the SmartSense Gateway is initiated by the server, using the following ports:

Source ComponentDestination Component Destination Port Transport SecurityPurpose
HST ServerSmartSense Gatewaytcp/9450One-way SSLHST server Registration
HST ServerSmartSense Gatewaytcp/9451Two-way SSLEncrypted bundle transfer

HST servers register themselves with the SmartSense Gateway using the two-way SSL registration port (tcp/9451), and when bundle capture is complete, this port is used to securely stream the bundle file to the SmartSense Gateway.

SmartSense Gateway to Hortonworks

Hortonworks does not initiate communications to the SmartSense Gateway, all communication is initiated by the SmartSense Gateway to Hortonworks. For this interaction, the following ports are used:

Source ComponentDestination Component Destination Port Purpose
GatewayHortonworkstcp/443HTTPS bundle upload

As bundles are captured, the HST server uses the two-way SSL communication channel to securely stream the bundle file to the SmartSense Gateway. After this process starts, the SmartSense Gateway opens up a secure connection to Hortonworks, using the HTTPS port to upload the bundle.

You have two options when configuring the communication between the SmartSense Gateway and Hortonworks:

  • Allow firewall access from the Gateway to a CNAME using port 443.

    The Hortonworks HTTPS servers utilize Elastic Load Balancing from Amazon Web Services. The CNAME is recommended as the number of instances, and IPs of instances used by the load balancer are fluid. Using the CNAME provides the greatest availability.

  • Allow firewall access from the Gateway to a pair of static IPs using port 443.

    These IPs do not change, and they use "round-robin" DNS for load balancing. This is the least preferred option, because instance availability is not quickly updated in DNS.

Details about both options are available at the following location: