1.1.7. Modify HDFS Configurations

You may need to modify your hdfs-site configuration and/or your core-site configuration.

  1. To check if you need to modify your hdfs-site configuration, on the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PW -port $AMBARI_PORT get localhost $CLUSTER_NAME hdfs-site

    If you see any of the following properties, you must delete them from your hdfs-site configuration.

    • dfs.nameservices

    • dfs.client.failover.proxy.provider.${NAMESERVICE_ID}

    • dfs.ha.namenodes.${NAMESERVICE_ID}

    • dfs.ha.fencing.methods

    • dfs.ha.automatic-failover.enabled

    • dfs.namenode.http-address.${NAMESERVICE_ID}.nn1

    • dfs.namenode.http-address.${NAMESERVICE_ID}.nn2

    • dfs.namenode.rpc-address.${NAMESERVICE_ID}.nn1

    • dfs.namenode.rpc-address.${NAMESERVICE_ID}.nn2

    • dfs.namenode.shared.edits.dir

    • dfs.journalnode.edits.dir

    • dfs.journalnode.http-address

    • dfs.journalnode.kerberos.internal.spnego.principal

    • dfs.journalnode.kerberos.principal

    • dfs.journalnode.keytab.file

      Where ${NAMESERVICE_ID} is the NameService ID you created when you ran the Enable NameNode HA wizard.

  2. To delete these properties, execute the following for each property you found. On the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PW -port $AMBARI_PORT delete localhost $CLUSTER_NAME hdfs-site property_name

    Where you replace property_name with the name of each of the properties to be deleted.

  3. Verify that all of the properties have been deleted. On the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PW -port $AMBARI_PORT get localhost $CLUSTER_NAME hdfs-site

    None of the properties listed above should be present.

  4. To check if you need to modify your core-site configuration, on the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PW -port $AMBARI_PORT get localhost $CLUSTER_NAME core-site
  5. If you see the property ha.zookeeper.quorum, it must be deleted. On the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PW -port $AMBARI_PORT delete localhost $CLUSTER_NAME core-site ha.zookeeper.quorum
  6. If the property fs.defaultFS is set to the NameService ID, it must be reverted back to its non-HA value. For example:

    “fs.defaultFS” : “hdfs://name-service-id”
    The property fs.defaultFS needs to be modified as it points to a NameService ID
    
    “fs.defaultFS” : “hdfs://nn01.mycompany.com” 
    The property fs.defaultFS does not need to be changed as it points to a specific NameNode and not a NameService ID
  7. To revert the property fs.defaultFS to the NameNode host value, on the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PW -port $AMBARI_PORT set localhost $CLUSTER_NAME core-site fs.defaultFS hdfs://${NAMENODE_HOSTNAME}
  8. Verify that the core-site properties are now properly set. On the Ambari Server host:

    /var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PW -port $AMBARI_PORT get localhost $CLUSTER_NAME core-site

    The property fs.defaultFS should be set to point to the NameNode host and the property ha.zookeeper.quorum should not be there.


loading table of contents...