Files and Directories
Persistence of HDFS metadata is implemented using fsimage file and edits files.
Do not attempt to modify metadata directories or files. Unexpected modifications can cause HDFS downtime, or even permanent data loss. This information is provided for educational purposes only.
Persistence of HDFS metadata broadly consist of two categories of files:
Contains the complete state of the file system at a point in time. Every file system modification is assigned a unique, monotonically increasing transaction ID. An
fsimagefile represents the file system state after all modifications up to a specific transaction ID.
- edits file
Contains a log that lists each file system change (file creation, deletion or modification) that was made after the most recent
Checkpointing is the process of merging the content of the most recent
fsimage, with all
edits applied after that
fsimage is merged, to create a new
Checkpointing is triggered automatically by configuration policies or manually by
HDFS administration commands.