HDFS Administration
Also available as:
PDF
loading table of contents...

Recommended Configurations

This section provides recommended configurations for Background and Fast modes.

The HDFS Balancer can run in either Background or Fast modes.

Background Mode

HDFS Balancer runs as a background process. The cluster serves other jobs and applications at the same time.

Fast Mode

HDFS Balancer runs at maximum (fast) speed.

Table 5.2. Datanode Configuration Properties

PropertyDefaultBackground ModeFast Mode
dfs.datanode.balance.max.concurrent.moves54 x (# of disks)4 x (# of disks)
dfs.datanode.balance.max.concurrent.moves1048576 (1 MB)use default10737418240 (10 GB)

Table 5.3. Balancer Configuration Properties

PropertyDefaultBackground ModeFast Mode
dfs.datanode.balance.max.concurrent.moves5# of disks4 x (# of disks)
dfs.balancer.moverThreads1000use default20,000
dfs.balancer.max-size-to-move10737418240 (10 GB)1073741824 (1GB)107374182400 (100 GB)
dfs.balancer.getBlocks.min-block-size10485760 (10 MB)use default104857600 (100 MB)