If you get the error "Cannot start datanode because the configured max locked memory size... is more than the datanode's available RLIMIT_MEMLOCK ulimit," that means that the operating system is imposing a lower limit on the amount of memory that you can lock than what you have configured.
To fix this, you must adjust the ulimit -l
value that the DataNode
runs with. This value is usually configured in
/etc/security/limits.conf
, but this may vary depending on what
operating system and distribution you are using.
You will know that you have correctly configured this value when you can
run ulimit -l
from the shell and get back either a higher value
than what you have configured with dfs.datanode.max.locked.memory
, or the string "unlimited",
which indicates that there is no limit. Note that it is typical for ulimit
-l
to output the memory lock limit in kilobytes (KB),
but dfs.datanode.max.locked.memory
must be specified in
bytes.
For example, if the value of dfs.datanode.max.locked.memory
is set to
128000 bytes:
<property> <name>dfs.datanode.max.locked.memory</name> <value>128000</value> </property>
You would set the memlock
(max locked-in-memory address space) to a
slightly higher value. For example, to set memlock
to 130 KB (130,000
bytes) for the hdfs
user, you would add the following line to
/etc/security/limits.conf
.
hdfs - memlock 130