Apache Ambari Upgrade
Also available as:
PDF

Rolling Upgrade Prerequisites

To perform a Rolling Upgrade, your cluster must meet the following prerequisites. If your cluster does not meet these upgrade prerequisites, you can consider an Express Upgrade.

HDFS

NameNode HA

NameNode HA must be enabled and working properly. From Ambari Web, the NameNodes, ZKFailoverControllers and JournalNodes must be “green” and showing no alerts.

NameNode Truncate

In HDP 2.2, truncate must be disabled and this prerequisite is REQUIRED. The hdfs-site configuration dfs.allow.truncate, property is set to true if truncate is enabled. If the property is not set or is false, truncate is disabled.

In HDP 2.3 and later, truncate is always enabled, which is acceptable. Therefore, this check is not applicable and will not be performed with HDP 2.3 maintenance upgrades.

YARN

Timeline Service State Recovery

YARN state recovery should be enabled. Check the Services > YARN > Configs > Advanced property yarn.timeline-service.recovery.enabled is set to true.

ResourceManager Work Preserving Recovery

YARN work preserving recovery should be enabled. Check the Services > YARN > Configs > Advanced property yarn.resourcemanager.work-preserving-recovery.enabled is set to true.

ResourceManager HA

ResourceManager HA should be enabled to prevent a disruption in service during the upgrade.

This prerequisite is OPTIONAL.

MapReduce2

MapReduce Distributed Cache

MapReduce should reference Hadoop libraries from the distributed cache in HDFS.

JobHistory State Recovery

JobHistory state recovery should be enabled. Check the following: Services > MapReduce > Configs > Advanced property mapreduce.jobhistory.recovery.enable is set to true . Once enabled, dependent properties for mapreduce.jobhistory.recovery.store.class and mapreduce.jobhistory.recovery.store.leveldb.path should also be set.

Encrypted Shuffle

If encrypted shuffle has been enabled, the ssl-client.xml file must be copied to /etc/hadoop/conf/secure directory on each node in the cluster.

Tez

Tez Distributed Cache

Tez should reference Hadoop libraries from the distributed cache in HDFS. Check the Services > Tez > Configs configuration and confirm tez.lib.uris is set with the path in HDFS for the tez.tar.gz file.

Hive

Hive Dynamic Service Discovery

HiveServer2 dynamic service discovery should be configured for Rolling Upgrade.

Hive Client Retry

Hive client retry properties must be configured. Review the Services > Hive > Configs configuration and confirm hive.metastore.failure.retries and hive.metastore.client.connect.retry.delay are specified.

Multiple Hive Metastore

Multiple Hive Metastore instances are recommended for Rolling Upgrade. This ensures that there is at least one Hive Metastore running during the upgrade process. Additional Hive Metastore instances can be added through Ambari.

This prerequisite is OPTIONAL.

Oozie

Oozie Client Retry

Oozie client retry properties must be configured. Review the Services > Oozie > Configs > oozie-env configuration and confirm "export OOZIE_CLIENT_OPTS="${OOZIE_CLIENT_OPTS} -Doozie.connection.retry.count=<number of retries>" is specified.

Druid

Druid High Availability

Enable Druid high availabilty (HA) before performing a rolling upgrade. Multiple instances of Druid services running in the same cluster avoid downtime during the upgrade. If you choose not to enable HA for Druid services, a warning message appears but you can start the upgrade with the risk that Druid might encounter downtime.

More Information

Configuring NameNode High Availability.

ResourceManager High Availability.

YARN Resource Management