Accessing Cloud Data
Also available as:
PDF
loading table of contents...

Embedding Credentials in the URL to Authenticate (Deprecated)

Embedding credentials in the URL is dangerous and deprecated. Due to the security risk it represents, future versions of Hadoop may remove this feature entirely. Use per-bucket configuration options instead.

Hadoop supports embedding credentials within the S3 URL:

s3a://key:secret@bucket-name/

In general, we strongly discourage using this mechanism, as it invariably results in the secret credentials being logged in many places in the cluster. However, embedding credentials in the URL is sometimes useful when troubleshooting authentication problems; consult the troubleshooting documentation for details.

Before S3A supported per-bucket credentials, this was the sole mechanism for supporting different credentials for different buckets. Now that buckets can be individually configured, this mechanism should no longer be needed. You should use per-bucket configuration options instead.