Launching Cloudbreak on GCP
These steps describe how to launch Cloudbreak on GCP for production. Before launching Cloudbreak on Google Cloud, review and meet the prerequisites. Next, follow the steps below.
To launch the Cloudbreak deployer and install the Cloudbreak application, you must have an existing VM.
Your system must meet the following requirements:
- Minimum VM requirements: 16GB RAM, 40GB disk, 4 cores
- Supported operating systems: RHEL, CentOS, and Oracle Linux 7 (64-bit)
You can install Cloudbreak on Mac OS X for evaluation purposes only. Mac OS X is not supported for a production deployment of Cloudbreak.
Every command must be executed as root. In order to get root privileges execute:
Ensure that your system is up-to-date by executing:
yum -y update
Reboot it if necessary.
Perform these steps to install and configure iptables.
yum -y install net-tools ntp wget lsof unzip tar iptables-services systemctl enable ntpd && systemctl start ntpd systemctl disable firewalld && systemctl stop firewalld
Without iptables-services installed the
iptables savecommand will not be available.
Configure permissive iptables on your machine:
iptables --flush INPUT && \ iptables --flush FORWARD && \ service iptables save
Perform these steps to disable SELINUX.
setenforce 0 sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
Run the following command to ensure that SELinux is not turned on afterwards:
The command should return "Disabled".
Perform these steps to install Docker. The minimum Docker version is 1.13.1. If you are using an older image that comes with an older Docker version, upgrade Docker to 1.13.1 or newer.
Install Docker service:
yum install -y docker systemctl start docker systemctl enable docker
yum install yum-utils yum-config-manager --enable rhui-REGION-rhel-server-extras yum install -y docker systemctl start docker systemctl enable docker
Check the Docker Logging Driver configuration:
docker info | grep "Logging Driver"
If it is set to
Logging Driver: journald, you must set it to "json-file" instead. To do that:
dockerfile for editing:
Edit the following part of the file so that it looks like below (showing
# Modify these options if you want to change the way the docker daemon runs OPTIONS='--selinux-enabled --log-driver=json-file --signature-verification=false'
systemctl restart docker systemctl status docker
Install Cloudbreak on a VM
Install Cloudbreak using the following steps.
Install the Cloudbreak deployer and unzip the platform-specific single binary to your PATH. For example:
yum -y install unzip tar curl -Ls public-repo-1.hortonworks.com/HDP/cloudbreak/cloudbreak-deployer_2.7.2_$(uname)_x86_64.tgz | sudo tar -xz -C /bin cbd cbd --version
Once the Cloudbreak deployer is installed, you can set up the Cloudbreak application.
Create a Cloudbreak deployment directory and navigate to it:
mkdir cloudbreak-deployment cd cloudbreak-deployment
In the directory, create a file called
Profilewith the following content:
export UAA_DEFAULT_SECRET=MY-SECRET export UAA_DEFAULT_USER_PW=MY-PASSWORD export UAA_DEFAULT_USER_EMAIL=MY-EMAIL export PUBLIC_IP=MY_VM_IP
export UAA_DEFAULT_SECRET=MySecret123 export UAA_DEFAULT_USER_PW=MySecurePassword123 export UAA_DEFAULT_USER_EMAILfirstname.lastname@example.org export PUBLIC_IP=172.26.231.100
You will need to provide the email and password when logging in to the Cloudbreak web UI and when using the Cloudbreak CLI. The secret will be used by Cloudbreak for authentication.
You should set the CLOUDBREAK_SMTP_SENDER_USERNAME variable to the username you use to authenticate to your SMTP server. You should set the CLOUDBREAK_SMTP_SENDER_PASSWORD variable to the password you use to authenticate to your SMTP server.
Generate configurations by executing:
rm *.yml cbd generate
The cbd start command includes the cbd generate command which applies the following steps:
- Creates the
docker-compose.ymlfile, which describes the configuration of all the Docker containers required for the Cloudbreak deployment.
- Creates the
uaa.ymlfile, which holds the configuration of the identity server used to authenticate users with Cloudbreak.
- Creates the
Start the Cloudbreak application by using the following commands:
cbd pull-parallel cbd start
This will start the Docker containers and initialize the application. The first time you start the Cloudbreak app, the process will take longer than usual due to the download of all the necessary docker images.
If you encounter errors during
cbd start, refer to Toubleshooting.
Next, check Cloudbreak application logs:
cbd logs cloudbreak
You should see a message like this in the log:
Started CloudbreakApplication in 36.823 seconds.Cloudbreak normally takes less than a minute to start.
Access Cloudbreak web UI
Log in to the Cloudbreak UI using the following steps.
You can log into the Cloudbreak application at
https://IP_Address. For example
https://22.214.171.124. You may use
cbd startto obtain the login information. Alternatively, you can obtain the VM's IP address from your cloud provider console.
Confirm the security exception to proceed to the Cloudbreak web UI.
The first time you access Cloudbreak UI, Cloudbreak will automatically generate a self-signed certificate, due to which your browser will warn you about an untrusted connection and will ask you to confirm a security exception.
Browser Steps Firefox Click Advanced > Click Add Exception... > Click Confirm Security Exception Safari Click Continue Chrome Click Advanced > Click Proceed...
The login page is displayed:
Log in to the Cloudbreak web UI using the credentials that you configured in your
- The username is the
- The password is the
- The username is the
Upon a successful login, you are redirected to the dashboard:
Configure external Cloudbreak database
By default, Cloudbreak uses an embedded PostgreSQL database to persist data related to Cloudbreak configuration, setup, and so on. For a production Cloudbreak deployment, you must configure an external database.
Configure an external database
Create Cloudbreak credential
Cloudbreak works by connecting your GCP account through this credential, and then uses it to create resources on your behalf. Before you can start provisioning cluster using Cloudbreak, you must create a Cloudbreak credential.
In order to launch clusters on GCP via Cloudbreak, you must have a service account that Cloudbreak can use to create resources. If you do not have one, refer to Prerequisites: Service account.
In the Cloudbreak web UI, select Credentials from the navigation pane.
Click Create Credential.
Under Cloud provider, select "Google Cloud Platform":
Provide the following information:
Parameter Description Key type Select JSON or P12. Since activating service accounts with P12 private keys has been deprecated in the Cloud SDK, we recommend using JSON. Name Enter a name for your credential. Description (Optional) Enter a description. Project Id (Only required for P12 key type) Enter the project ID. You can obtain it from your GCP account by clicking on the name of your project at the top of the page and copying the ID. Service Account Email Address (Only required for P12 key type) "Service account ID" value for your service account created in prerequisites. You can find it on GCP at IAM & Admin > Service accounts. Service Account Private Key Upload the key that you created in the prerequisites when creating a service account.
Your credential should now be displayed in the Credentials pane.
Congratulations! You have successfully launched Cloudbreak and created a Cloudbreak credential. Now you can use Cloudbreak to create clusters.