Hortonworks Docs » Data Lifecycle Manager 1.1.2 » DLM Administration
DLM Administration
Also available as:
PDF
Next

HDFS replication policy process overview

  1. The DLM App submits the replication policy to the DLM Engine on the destination cluster. The DLM Engine then schedules replication jobs at the specified frequency.
  2. At the specific frequency, DLM Engine submits a DistCp job that runs on destination YARN, reads data from source HDFS, and writes to destination HDFS.
  3. File length and checksums are used to determine changed files and validate that the data is copied correctly.

  4. The Ranger policies for the HDFS directory are exported from source Ranger service and replicated to destination Ranger service.

    DLM Engine also adds a deny policy on the destination Ranger service for the target directory so that the target is not writable.

© 2012–2019, Hortonworks, Inc.
Document licensed under the Creative Commons Attribution ShareAlike 4.0 License.
Hortonworks.com | Documentation | Support | Community