DocsHortonworks Data Platform

 

 3.2. Test LDAP Connection

Test the LDAP connection use a curl command from the Knox Gateway to lists the contents of the directory tmp/test (or use a directory that exists in your environment):

curl -i -k -u $ldap_user:$password -X GET \    
'https://$gateway_host:8443/$gateway_path/$cluster_name/webhdfs/api/v1/tmp/test?op=LISTSTATUS'

where the variables in the above command match actual items in your environment:

  • $ldap_user and $password is an actual user account in your environment (the LDAP provider binds to the UserDN).

  • $gateway_host and $gateway_path matches your gateway configuration.

  • $cluster_name matches the name of the cluster topology descriptor file of the cluster you are trying to access.

Example outputs related to authentication messages:

  • Successful Authentication. The following requests and logs show normal operation:

    • Client-side successful request to WebHDFS using HDP Sandbox, the Knox sample Apache Directory, and the Knox Test Account guest:

      curl -i -k -u guest:guest-password -X GET 'https://sandbox:8443/gateway/sandbox/webhdfs/v1/user/?op=LISTSTATUS'
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=2hf99emf1dr31mzhjmpimwf1w;Path=/gateway/sandbox;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache
Expires: Fri, 06 Jun 2014 20:26:53 GMT
Date: Fri, 06 Jun 2014 20:26:53 GMT
Pragma: no-cache
Expires: Fri, 06 Jun 2014 20:26:53 GMT
Date: Fri, 06 Jun 2014 20:26:53 GMT
Pragma: no-cache
Server: Jetty(6.1.26)
Content-Type: application/json
Content-Length: 1515

{"FileStatuses":{"FileStatus":[{"accessTime":0,"blockSize":0,"childrenNum":9,"fileId":16388,"group":"hdfs","length":0,
"modificationTime":1398090362268,"owner":"ambari-qa","pathSuffix":"ambari-qa","permission":"770","replication":0,"type"
:"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":17117,"group":"guest","length":0,"modificationTime"
:1398176498899,"owner":"guest","pathSuffix":"guest","permission":"755","replication":0,"type":"DIRECTORY"},{"accessTime"
:0,"blockSize":0,"childrenNum":0,"fileId":16949,"group":"hdfs","length":0,"modificationTime":1398090186250,"owner":"hcat",
"pathSuffix":"hcat","permission":"755","replication":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":0,
"fileId":16418,"group":"hdfs","length":0,"modificationTime":1398089829227,"owner":"hive","pathSuffix":"hive","permission":
"700","replication":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":3,"fileId":17019,"group":"hue","length"
:0,"modificationTime":1398176493665,"owner":"hue","pathSuffix":"hue","permission":"755","replication":0,"type":"DIRECTORY"},
{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16441,"group":"hdfs","length":0,"modificationTime":1398089926561,
"owner":"oozie","pathSuffix":"oozie","permission":"775","replication":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,
"childrenNum":0,"fileId":17012,"group":"root","length":0,"modificationTime":1398176445256,"owner":"root","pathSuffix":"root",
"permission":"755","replication":0,"type":"DIRECTORY"}]}}[root@sandbox xasecure-ha

      The output above shows a success authentication and returns the content list.

    • Client-side request with WebHDFS error. Using the Knox sample environment on Sandbox. The user authenticates and requests a list from WebHDFS on a directory does not exist:

      curl -i -k -u guest:guest-password -X GET 'https://sandbox:8443/gateway/sandbox/webhdfs/v1/user/myuser?op=LISTSTATUS'
HTTP/1.1 404 Not Found
Set-Cookie: JSESSIONID=10n9m664r778o1otump5s3jjqb;Path=/gateway/sandbox;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache
Expires: Fri, 06 Jun 2014 20:57:26 GMT
Date: Fri, 06 Jun 2014 20:57:26 GMT
Pragma: no-cache
Expires: Fri, 06 Jun 2014 20:57:26 GMT
Date: Fri, 06 Jun 2014 20:57:26 GMT
Pragma: no-cache
Server: Jetty(6.1.26)
Content-Type: application/json
Content-Length: 151

{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException",
"message":"File /user/myuser does not exist."}}[ro...

    • Server Log examples. Each time a user tries to access a Hadoop REST API through the Knox Gateway, the attempt is captured in the audit log. For example:

      14/06/06 13:26:53 ||c05b42e8-06aa-4673-959e-37743573dfdc|audit|WEBHDFS|guest|||access|uri|/gateway/sandbox/webhdfs/v1/user/?op=LISTSTATUS|success|Response st
atus: 200
14/06/06 13:31:45 ||d63cb431-9cb9-4bd4-8d8c-26202dd1a71f|audit|WEBHDFS|guest|||access|uri|/gateway/sandbox/webhdfs/v1/user/?op=LISTSTATUS|unavailable|
14/06/06 13:57:26 ||0409c32a-2b29-4a05-8798-0420682e930b|audit|WEBHDFS||||access|uri|/gateway/sandbox/webhdfs/v1/user/myuser?op=LISTSTATUS|unavailable|
14/06/06 13:57:26 ||0409c32a-2b29-4a05-8798-0420682e930b|audit|WEBHDFS|guest|||authentication|uri|/gateway/sandbox/webhdfs/v1/user/myuser?op=LISTSTATUS|succe
ss|
14/06/06 13:57:26 ||0409c32a-2b29-4a05-8798-0420682e930b|audit|WEBHDFS|guest|||dispatch|uri|http://sandbox.hortonworks.com:50070/webhdfs/v1/user/myuser?user.
name=guest&op=LISTSTATUS|success|Response status: 404
14/06/06 13:57:26 ||0409c32a-2b29-4a05-8798-0420682e930b|audit|WEBHDFS|guest|||access|uri|/gateway/sandbox/webhdfs/v1/user/myuser?op=LISTSTATUS|success|Response status: 404

  • Authentication failures. When the user cannot be authenticated, the request is rejected with an HTTP status of 401 unauthorized regardless of the reason. The client receives the same error message (shown below) in the case of invalid credentials and if the Knox Gateway CANNOT establish a connection to the LDAP service:

    curl -i -k -u guest:password -X GET 'https://sandbox:8443/gateway/sandbox/webhdfs/v1/user/?op=LISTSTATUS'
HTTP/1.1 401 Unauthorized
WWW-Authenticate: BASIC realm="application"
Content-Length: 0
Server: Jetty(8.1.14.v20131031)

    To verify connectivity to the LDAP service, enable debug on the Shiro provider and check the logs.

    • Log example for LDAP service connection issues. The gateway log with the Shiro provider debug enable contains an error messages similar to the ones below:

      2014-06-06 16:06:55,831 DEBUG authc.BasicHttpAuthenticationFilter (BasicHttpAuthenticationFilter.java:createToken(308)) - Attempting to execute login with headers [Basic Z3Vlc3Q6Z3Vlc3QtcGFzc3dvcmQ=]
2014-06-06 16:06:55,832 DEBUG ldap.JndiLdapRealm (JndiLdapRealm.java:queryForAuthenticationInfo(369)) - Authenticating user 'guest' through LDAP
2014-06-06 16:06:55,832 DEBUG ldap.JndiLdapContextFactory (JndiLdapContextFactory.java:getLdapContext(488)) - Initializing LDAP context using URL [ldap://localhost:33389] and principal [uid=guest,ou=people,dc=hadoop,dc=apache,dc=org] with pooling disabled
2014-06-06 16:06:55,838 DEBUG servlet.SimpleCookie (SimpleCookie.java:addCookieHeader(226)) - Added HttpServletResponse Cookie [rememberMe=deleteMe; Path=/gateway/sandbox; Max-Age=0; Expires=Thu, 05-Jun-2014 23:06:55 GMT]
2014-06-06 16:06:55,839 DEBUG authc.BasicHttpAuthenticationFilter (BasicHttpAuthenticationFilter.java:sendChallenge(274)) - Authentication required: sending 401 Authentication challenge response.

    • Log example for invalid credentials:

      14/06/06 13:31:45 ||d63cb431-9cb9-4bd4-8d8c-26202dd1a71f|audit|WEBHDFS|guest|||access|uri|/gateway/sandbox/webhdfs/v1/user/?op=LISTSTATUS|success|Response st
atus: 401
Legal notices
loading table of contents...