5.3. OS Limits

If you get the error "Cannot start datanode because the configured max locked memory size... is more than the datanode's available RLIMIT_MEMLOCK ulimit," that means that the operating system is imposing a lower limit on the amount of memory that you can lock than what you have configured.

To fix this, you must adjust the ulimit -l value that the DataNode runs with. This value is usually configured in /etc/security/limits.conf, but this may vary depending on what operating system and distribution you are using.

You will know that you have correctly configured this value when you can run  ulimit -l from the shell and get back either a higher value than what you have configured with dfs.datanode.max.locked.memory, or the string "unlimited", which indicates that there is no limit. Note that it is typical for  ulimit -l to output the memory lock limit in kilobytes (KB), but dfs.datanode.max.locked.memory must be specified in bytes.

For example, if the value of dfs.datanode.max.locked.memory is set to 128000 bytes:

<property>
    <name>dfs.datanode.max.locked.memory</name>
    <value>128000</value>
  </property>

You would set the memlock (max locked-in-memory address space) to a slightly higher value. For example, to set memlock to 130 KB (130,000 bytes) for the hdfs user, you would add the following line to /etc/security/limits.conf.

hdfs             -       memlock         130


loading table of contents...