4.1. distCP Throttle

Falcon uses distcp (distributed copy) for data replication. If you need to optimize bandwidth between data centers, you can throttle bandwidth during Falcon data replication as needed and limit the number of maps used during replication.

To throttle distcp:

  1. If you already have Falcon running on your clusters, suspend your current active feeds and processes:

    $FALCON_HOME/bin/falcon entity -type $feedName -name $name -suspend 
    $FALCON_HOME/bin/falcon entity -type $processName -name $name -suspend 
  2. Edit your feed entity or entities. Add the following lines:

    <properties>
       <property name="maxMaps" value="$integerValue" />
       <property name="maxBandwidth" value="$MB/svalue" />
    </properties>                  
    [Note]Note

    Specify the maximum number of mappers for Falcon to use in maxMaps. Specify the bandwidth in MB for each mapper in mapBandwidth.

  3. Sumbit your updated feed entity.

    $FALCON_HOME/bin/falcon entity -submit -type feed -file ~$feedFileName
  4. Resume your processes.

    $FALCON_HOME/bin/falcon entity -type $processName -name $name -resume
    $FALCON_HOME/bin/falcon entity -type $feedName -name $name -resume