Skip to content

Instantly share code, notes, and snippets.

@tuliren

tuliren/README.md

Last active March 22, 2019 06:42
Show Gist options
  • Save tuliren/03bf8844087e30a9bd20f95187d553c3 to your computer and use it in GitHub Desktop.
Save tuliren/03bf8844087e30a9bd20f95187d553c3 to your computer and use it in GitHub Desktop.
Download ZIP
Hadoop Memory Configuration
Raw
README.md
Parameter Purpose Default
yarn.app.mapreduce.am.resource.mb The physical memory requirement, in MiB, for the ApplicationMaster. 1 GiB
mapreduce.map.memory.mb The amount of physical memory, in MiB, allocated for each map task of a job. 1 GiB
mapreduce.reduce.memory.mb The amount of physical memory, in MiB, allocated for each reduce task of a job. 1 GiB
mapreduce.map.java.opts Java opts for the map processes. -Djava.net.preferIPv4Stack=true
mapreduce.reduce.java.opts Java opts for the reduce processes. -Djava.net.preferIPv4Stack=true
mapred.child.java.opts Java opts for the map processes. It has been deprecated in Hadoop 2.0. -Djava.net.preferIPv4Stack=true
yarn.nodemanager.vmem-pmem-ratio Virtual / physical memory ratio. Virtual memory is allocated based on this ratio 2.1

Reference:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment