Cloud Data Access
Also available as:
PDF
loading table of contents...

Configuring Access to Google Cloud Storage

Access from a cluster to a Google Cloud Storage is possible through a service account. Configuring access to Google Cloud Storage involves the following steps.

Table 6.1. Overview of Configuring Access to Google Cloud Storage

StepConsiderations
Creating a service account on Google Cloud Platform and generating a key associated with it.
  • You may need to contact your Google Cloud Platform admin in order to complete these steps.

  • If you already have a service account, you do not need to perform these steps as long as you are able to provide the service account key. If you have a service account but do not know the service account key, you should be able to generate a new key.

Modifying permissions of the Google Cloud Storage bucket so that you can access it by using your service account key.
  • You may need to contact your Google Cloud Platform admin in order to complete these steps.

  • You will typically perform these steps for each bucket that you want to access.

  • You do not need to perform these steps if your service account has project-wide access to all buckets on the account.

Placing the service account key on all nodes of the cluster and setting related properties in Ambari.
  • These configuration steps are appropriate for a single-user cluster.

  • Only one configuration per cluster is recommended, that is, you should use one service account key per cluster. If required, it is possible to use multiple service account keys with the same cluster; In this case, multiple service account keys should be available on all nodes, and each job-specific configuration should use one selected key.