Tutorial: How to Launch HDCloud Controller via AWS CLI

This tutorial shows you how to use the AWS Command Line Interface to launch the Hortonworks Data Cloud (HDCloud) for AWS cloud controller.

Overview

In this tutorial, we will use the AWS CLI to launch the HDCloud cloud controller. The steps include:

  1. Meeting the prerequisites.

  2. Subscribing to HDCloud AWS Marketplace services.

  3. Launching cloud controller via AWS CLI (basic template).

  4. Adding additional parameters if using the advanced template.

Our suggestion is that you first go through the launch process using the CloudFormation Management Console to get familiar with the procedure. Once you're comfortable with the launch process, you can automate via the AWS CLI. For instructions on how to launch using the UI, refer to the Launch Cloud Controller documentation.

Meet the Prerequisites

  1. Log in to your AWS account. If you don’t have one, set it up and then generate or upload your SSH keypair.

  2. Install and configure the AWS CLI tool. For links and instructions, refer to AWS documentation, including:

    1. Install - Download and install AWS CLI.
    2. Create Access Keys - Create access keys necessary to configure the AWS CLI.
    3. Configure - Configure access keys, default region, and default output format.
  3. In addition to the basic configuration, you will need to add policies that will allow your user to create resources. This includes AWS managed policies and custom policies.

    Here are the policies that I used:

    I added the following custom policies via AWS Policy Generator:

Subscribe

Subscribe to the following Hortonworks Data Cloud AWS Marketplace services:

Launch Cloud Controller via AWS CLI

Submit the request to create stack using the AWS CLI. Here is the template:

aws cloudformation create-stack \
--capabilities CAPABILITY_IAM \
--parameters \
ParameterKey=KeyName,ParameterValue=MY_KEY_NAME \
ParameterKey=EmailAddress,ParameterValue=MY_EMAIL \
ParameterKey=AdminPassword,ParameterValue=MY_PASSWORD \
ParameterKey=RemoteLocation,ParameterValue=REMOTE_ACCESS \
ParameterKey=InstanceType,ParameterValue=INSTANCE_TYPE \
ParameterKey=ProductTelemetry,ParameterValue="TELEMETRY" \
--stack-name "MY_STACK_NAME" \
--disable-rollback \
--template-body https://s3.amazonaws.com/awsmp-fulfillment-cf-templates-prod/571fb43d-99f6-4182-8166-61c477473f09.0aafa7fc-a77d-4bea-82e9-abd3eb5bed6a.template

The following should be replaced with your chosen values:

MY_KEY_NAME: Enter the name of your private SSH key used in the chosen region
MY_STACK_NAME: Choose a name for your stack
MY_PASSWORD: Choose a password
MY_EMAIL: Enter a valid email address
REMOTE_ACCESS: Specify a valid CIDR range. You can use this tool to calculate a valid CIDR range that includes your public IP address. Or, at your own risk, you can enter 0.0.0.0/0 to allow access from all.

The following parameters are optional. If you want to use default values, you can simply delete the lines that declare these parameters:

INSTANCE_TYPE: Default is m4.large
TELEMETRY: Default is “I Do Not Opt In to SmartSense Telemetry”. To opt in, use "I Have Read and Opt In to SmartSense Telemetry"

In case this is useful, here is the ‘create-stack` syntax reference documentation.

Here is an example:

aws cloudformation create-stack \
--capabilities CAPABILITY_IAM \
--parameters \
ParameterKey=KeyName,ParameterValue=dominika-kp-virg \
ParameterKey=EmailAddress,ParameterValue=dbialek@hortonworks.com \
ParameterKey=AdminPassword,ParameterValue=Password123! \
ParameterKey=RemoteLocation,ParameterValue=0.0.0.0/0 \
ParameterKey=InstanceType,ParameterValue=m4.large \
ParameterKey=ProductTelemetry,ParameterValue="I Have Read and Opt In to SmartSense Telemetry" \
--stack-name "domi-test" \
--disable-rollback \
--template-body https://s3.amazonaws.com/awsmp-fulfillment-cf-templates-prod/571fb43d-99f6-4182-8166-61c477473f09.0aafa7fc-a77d-4bea-82e9-abd3eb5bed6a.template

Check Status and More

To get status of the stack using the AWS CLI:

aws cloudformation describe-stacks --stack-name MY_STACK_NAME

For example:

aws cloudformation describe-stacks --stack-name domi-test

For more commands, refer to reference documentation.

Using the Advanced Template

The link in the example above points to the basic template. The following link points to the advanced template:

https://s3.amazonaws.com/awsmp-fulfillment-cf-templates-prod/571fb43d-99f6-4182-8166-61c477473f09.cc627713-56df-4208-8413-67af27a9f47a.template

Note that the advanced template requires using additional parameters:

aws cloudformation create-stack \
--capabilities CAPABILITY_IAM \
--parameters \
ParameterKey=KeyName,ParameterValue=MY_KEY_NAME \
ParameterKey=EmailAddress,ParameterValue=MY_EMAIL \
ParameterKey=AdminPassword,ParameterValue=MY_PASSWORD \
ParameterKey=RemoteLocation,ParameterValue=REMOTE_ACCESS \
ParameterKey=InstanceType,ParameterValue=INSTANCE_TYPE \
ParameterKey=ProductTelemetry,ParameterValue="TELEMETRY" \
ParameterKey=VPCID,ParameterValue=VPC_ID \
ParameterKey=SubnetID,ParameterValue=SUBNET_ID \
ParameterKey=RdsUrl,ParameterValue=RDS_ENDPOINT \
ParameterKey=RdsUsername,ParameterValue=RDS_USER \
ParameterKey=RdsPassword,ParameterValue=RDS_PASSWORD \
ParameterKey=RdsDatabase,ParameterValue=RDS_DB \
--stack-name "MY_STACK_NAME" \
--disable-rollback \
--template-body https://s3.amazonaws.com/awsmp-fulfillment-cf-templates-prod/571fb43d-99f6-4182-8166-61c477473f09.0a04baea-391d-4033-bcf2-1f7db8af07b2.template 

The following should be replaced with your chosen values:

VPC_ID: ID of an existing VPC
SUBNET_ID: ID of an existing subnet that is located within the specified VPC
RDS_ENDPOINT: URL:PORT for your existing RDS instance
RDS_USER: RDS instance user
RDS_PASSWORD: Password for your RDS instance user
RDS_DB: Name of a database that exists on your RDS instance

Here is an example:

aws cloudformation create-stack \
--capabilities CAPABILITY_IAM \
--parameters \
ParameterKey=KeyName,ParameterValue=dominika-kp-virg \
ParameterKey=EmailAddress,ParameterValue=dbialek@hortonworks.com \
ParameterKey=AdminPassword,ParameterValue=Password123! \
ParameterKey=RemoteLocation,ParameterValue=0.0.0.0/0 \
ParameterKey=InstanceType,ParameterValue=m4.large \
ParameterKey=ProductTelemetry,ParameterValue="I Have Read and Opt In to SmartSense Telemetry" \
ParameterKey=VPCID,ParameterValue=vpc-a90216ce \
ParameterKey=SubnetID,ParameterValue=subnet-52475f6f \
ParameterKey=RdsUrl,ParameterValue=domi-rds.crjdujkaybgv.us-east-1.rds.amazonaws.com:5432 \
ParameterKey=RdsUsername,ParameterValue=dbialek \
ParameterKey=RdsPassword,ParameterValue= MySecretPassword123 \
ParameterKey=RdsDatabase,ParameterValue=domidb \
--stack-name "domi-test" \
--disable-rollback \
--template-body https://s3.amazonaws.com/awsmp-fulfillment-cf-templates-prod/571fb43d-99f6-4182-8166-61c477473f09.0a04baea-391d-4033-bcf2-1f7db8af07b2.template

In case you are trying to use a different version of HDCloud than the one described or you want to verify the template link, you can obtain the template link (that is the value of template-body) using the following steps:

  1. From the AWS marketplace product listing page for Hortonworks Data Cloud - Controller Service, click Continue.
  2. On the next page, under Deployment Options, select BASIC or ADVANCED.
  3. Click Launch with CloudFormation Console.
  4. On the Create stack page in the CloudFormation console, you can find the template link: