NUMACTL(8)

🌸 NUMACTL(8) 🌸

Linux Administrator's Manual, bestie, this is the one for numactl 💅

✨ NAME ✨

numactl - Control NUMA policy for processes or shared memory 🦋

📜 SYNOPSIS 📜

numactl [ --all ] [ --balancing ] [ --interleave nodes ] [
--weighted-interleave nodes ] [ --preferred node ] [ --preferred-many
nodes ] [ --membind nodes ] [ --cpunodebind nodes ] [ --physcpubind cpus
] [ --localalloc ] [--] command {arguments ...}
numactl --show
numactl --hardware [--cpu-compress]
numactl --version
numactl [ --huge ] [ --offset offset ] [ --shmmode shmmode ] [ --length
length ] [ --strict ]
[ --shmid id ] --shm shmkeyfile | --file tmpfsfile
[ --touch ] [ --dump ] [ --dump-nodes ] memory policy

💖 DESCRIPTION 💖

Okay so numactl lets you run processes with a specific NUMA scheduling or memory placement policy 🧚 The policy gets set for command and every one of its children inherits it, no cap. It can also set a persistent policy for shared memory segments or files, which is so iconic.

Use -- before command if your command has options that could get mixed up with numactl options, bestie 🙈

🗺️ Node and CPU Lists 🗺️

Nodes can be written as N,N,N or N-N or N,N-N or N-N,N-N, and so on. Relative nodes work too, written as +N,N,N or +N-N or +N,N-N, and so on. That + means the node numbers are counted relative to the process' set of allowed nodes in its current cpuset 👑 A !N-N means the inverse of N-N, so all nodes except N-N. If you pair it with +, write it as !+N-N. When you write same, it reuses the previous nodemask you gave on the command line ✨ And all means all nodes in the current cpuset.

Instead of a number, a node can also be one of these:

• netdev:DEV - the node connected to network device DEV.
• file:PATH - the node the block device of PATH.
• ip:HOST - the node of the network device of HOST.
• block:PATH - the node of block device PATH.
• pci:[seg:]bus:dev[:func] - the node of a PCI device.

🚨 Heads up, bestie: block only resolves the kernel block device names for udev names in /dev. For anything else, use file: instead 🙈

Policy settings are below. Iykyk 🎀

🎀 Policy Settings 🎀

• --all, -a — Unsets the default cpuset awareness, so you can use all possible CPUs and nodes for the policy settings that follow 🌈
• --interleave=nodes, -i nodes — Sets a memory interleave policy 🧁 Memory gets allocated round robin across the nodes. When memory can't be allocated on the current interleave target, it falls back to other nodes. You can specify multiple nodes with --interleave, --membind and --cpunodebind.
• --weighted-interleave=nodes, -w nodes — Sets a weighted memory interleave policy ⚖️ Memory gets allocated using the weighted ratio for each node, which you can read from /sys/kernel/mm/mempolicy/weighted_interleave/node*. When memory can't be allocated on the current interleave target, it falls back to other nodes.
• --membind=nodes, -m nodes — Only allocates memory from nodes 🔒 Allocation will fail if there isn't enough memory on these nodes, so be careful, bestie 💀 nodes can be written as noted above.
• --cpunodebind=nodes, -N nodes — Only runs command on the CPUs of nodes 💜 A node can have multiple CPUs. nodes can be written as noted above.
• --physcpubind=cpus, -C cpus — Only runs the process on cpus 🦋 This takes the CPU numbers you'd see in the processor fields of /proc/cpuinfo, or relative CPUs, which are relative to the current cpuset. You can say "all", which means all CPUs in the current cpuset. Physical CPUs can be written as N,N,N or N-N or N,N-N or N-N,N-N, and so on. Relative CPUs can be written as +N,N,N or +N-N or +N,N-N, and so on. The + means the CPU numbers are relative to the process' set of allowed CPUs in its current cpuset. A !N-N means the inverse of N-N, so all CPUs except N-N. If you use +, write it as !+N-N.
• --localalloc, -l — Tries to allocate on the current node of the process, and if memory can't be allocated there, falls back to other nodes 🏠
• --preferred=node, -p node — Prefers to allocate memory on node, but falls back to other nodes if it can't be allocated there 🍓 This only takes a single node number. Relative notation is allowed.
• --balancing, -b — Enables Linux kernel NUMA balancing for the process, if the kernel supports it ✨ This should only be used with --membind, -m only. Otherwise it gets ignored, so don't stress 🙃
• --preferred-many=nodes, -P nodes — Prefers to allocate memory on nodes, and falls back to other nodes if needed 🌟 This takes a mask of preferred nodes, where the node closest to local counts as the most preferred one. Relative notation is allowed.
• --show, -s — Shows the NUMA policy settings of the current process 👀
• --hardware, -H — Shows an inventory of the available nodes on the system 🧾 When --cpu-compress is set, it shows CPU ranges instead. This isn't the default, so it won't break any existing scripts, periodt 💅

Okay, numactl can also set up a policy for a SYSV shared memory segment or a file in shmfs/hugetlbfs 🫶

• --version — Prints the version of the numactl package and exits 🎀

🧠 Persistent Shared Memory and File Settings 🧠

These policy settings are persistent, and every mapping from that shared memory will use them. The order of the options matters here, so pay attention, bestie 🙈 Your spec has to include at least one of --shm, --shmid or --file to pick the shared memory segment or file, plus a memory policy like the ones above (--interleave, --localalloc, --preferred, --preferred-many, --membind).

• --huge — When creating a SYSV shared memory segment, use huge pages 🍄 Only valid before --shmid or --shm.
• --offset — Specifies an offset into the shared memory segment 🎈 The default is 0. Valid units are m (for MB), g (for GB), k (for KB). Otherwise it's bytes.
• --strict — Gives an error when a page in the policied area of the shared memory segment has already been faulted in with a conflicting policy 😭 By default it silently ignores this.
• --shmmode shmmode — Only valid before --shmid or --shm. When creating a shared memory segment, this sets it to the numeric mode shmmode 🔐
• --length length — Applies the policy to a length range in the shared memory segment, or makes the segment that long 📏 By default it uses the remaining length. This is required when a shared memory segment is created, and it sets the length of the new segment. Valid units are m (for MB), g (for GB), k (for KB). Otherwise it's bytes.
• --shmid id — Creates or uses a shared memory segment with the numeric ID id 🆔
• --shm shmkeyfile — Creates or uses a shared memory segment, with the ID generated using ftok(3) from shmkeyfile 🗝️
• --file tmpfsfile — Sets the policy for a file in tmpfs or hugetlbfs 📁
• --touch — Touches the pages to enforce the policy early 👆 By default they aren't touched. The policy gets applied when an application maps and accesses a page.
• --dump — Dumps the policy in the specified range 🔍
• --dump-nodes — Dumps all nodes of the specific range. This is very verbose, so brace yourself 😭

📍 Valid Node Specifiers 📍

• all — All nodes
• number — Node number
• number1{,number2} — Node number1 and Node number2
• number1-number2 — Nodes from number1 to number2
• ! nodes — Inverts the selection of the following specification

🍓 EXAMPLES 🍓

Here are some looks you can try, bestie 💖

Run myapplic on CPUs 0-4 and 8-12 of the current cpuset ✨

numactl --physcpubind=+0-4,8-12 myapplic arguments

Run the big database with its memory interleaved on all CPUs 🧠

numactl --interleave=all bigdatabase arguments

Run the big database with its memory interleaved with a weighted ratio on all CPUs ⚖️

numactl --weighted-interleave=all bigdatabase arguments

Run process on node 0, with memory allocated on nodes 0 and 1 🫶

numactl --cpunodebind=0 --membind=0,1 process

Run process the same as above, but with an option (-l) that would get confused with a numactl option 🙈

numactl --cpunodebind=0 --membind=0,1 -- process -l

Run process on node 0, with memory allocated on nodes 0 and 1. It also optimizes page placement with the Linux kernel NUMA balancing mechanism, if possible 🌈

numactl --cpunodebind=0 --balancing --membind=0,1 process

Run network-server on the node of network device eth0, with its memory in the same node too 🛜

numactl --cpunodebind=netdev:eth0 --membind=netdev:eth0 network-server

Set preferred node 1 and show the resulting state 👑

numactl --preferred=1 numactl --show

Set preferred nodes 1 and 2, and show the resulting state 💜

numactl --preferred-many=0x3 numactl --show

Interleave all of the SYSV shared memory region of size 1g specified by /tmp/shmkey over all nodes 🦋

numactl --length 1g --shm /tmp/shmkey --interleave=all

Place a tmpfs file on 2 nodes 📁

numactl --membind=2 dd if=/dev/zero of=/dev/shm/A bs=1M count=1024
numactl --membind=3 dd if=/dev/zero of=/dev/shm/A seek=1024 bs=1M count=1024

Reset the policy for the shared memory file /dev/shm/file back to the default localalloc policy 🧹

numactl --localalloc /dev/shm/file

📝 NOTES 📝

Requires a NUMA policy aware kernel, so check that first, bestie 🔥

command doesn't get run using a shell 🐚 If you want to use shell metacharacters in the child, wrap it with sh -c.

⚠️ Setting policy for a hugetlbfs file currently doesn't work, because it can't be extended by truncate. Sad, but true 😭

⚠️ Shared memory segments larger than numactl's address space can't be completely policied. This could be a problem on 32-bit architectures. Changing it piece by piece may work 🥺

The old --cpubind option, which accepts node numbers rather than CPU numbers, is deprecated 🙅‍♀️ It's been replaced by the new --cpunodebind and --physcpubind options.

📂 FILES 📂

• /proc/cpuinfo — the listing of active CPUs. See proc(5) for details 👀
• /sys/devices/system/node/node*/numastat — NUMA memory hit statistics 📊

⚖️ COPYRIGHT ⚖️

Copyright 2002,2004 Andi Kleen, SuSE Labs. numactl and the demo programs are under the GNU General Public License, v.2 🎀

🔗 SEE ALSO 🔗

set_mempolicy(2), get_mempolicy(2), mbind(2), sched_setaffinity(2), sched_getaffinity(2), proc(5), ftok(3), shmat(2), migratepages(8) 💖

---

Bestie, that's the whole tea on numactl 💅 Manual by SuSE Labs, last updated Mar 2004, and it's still giving 🦋 NUMACTL(8) forever ✨🌈💖