Keeping Track of Logs
Replication of logs is done atomically and does not generate an exception when the reader is currently reading a log.
Each master cluster RegionServer has its own znode in the replication znodes hierarchy. It contains one znode per peer cluster. For example, if there are 5 slave clusters, 5 znodes are created, and each of these contain a queue of WALs to process. Each of these queues tracks the WALs created by that RegionServer, but they can differ in size. For example, if one slave cluster becomes unavailable for some time, the WALs should not be deleted. They need to stay in the queue while the others are processed.
When a source is instantiated, it contains the current WAL that the RegionServer is writing to. During log rolling, the new file is added to the queue of each slave cluster znode just before it is made available. This ensures that all the sources are aware that a new log exists before the RegionServer is able to append edits into it. However, this operation is now more expensive. The queue items are discarded when the replication thread cannot read more entries from a file because it reached the end of the last block and there are other files in the queue. This means that if a source is up to date and replicates from the log that the RegionServer writes to, reading up to the end of the current file does not delete the item in the queue.
A log can be archived if it
is no longer used or if the number of logs exceeds the
hbase.regionserver.maxlogs setting because the insertion rate is
faster than the rate at which the regions are flushed. When a log is archived, the
source threads are notified that the path for that log changed. If a particular source
has already finished with an archived log, it ignores the message. If the log is in the
queue, the path is updated in memory. If the log is currently being replicated, the
change is done atomically so that the reader does not attempt to open the file when it
has already been moved. Because moving a file is a NameNode operation, if the reader is
currently reading the log, it does not generate an exception.