When the VM starts, the HA Monitor waits for the NameNode to begin responding to file system operations. During this "bootstrap phase”, the HA monitor does not report startup failures of NameNode probes to the HA infrastructure. The HA monitor exits the bootstrap phase once all the probes succeed (from that point, the failure of a probe is reported as a service failure).
The time limit of the bootstrap phase can be configured using the service.monitor.bootstrap.timeout property:
<property> <name>service.monitor.bootstrap.timeout</name> <value>120000</value> <description> The time in milliseconds for the monitor to wait for the service to bootstrap and become available before it reports a failure to the management infrastructure </description> </property>
The timeout must be sufficiently long so that the monitored service is able to open its network ports for external interaction. For the NameNode, the web page and IPC port must be open.
The bootstrap time also needs to include the time required for the HDFS journal replay operations. The bootstrap timeout value should be kept high if the filesystem is large and if the secondary NameNode checkpointing time intervals are longer.
