numactl Command in Linux
The numactl command is a powerful tool for managing NUMA (Non-Uniform Memory Access) policies in Unix/Linux systems. The primary function of numactl is to control how processes are scheduled and where their memory is placed.
- By using the numactl command, you can specify that certain processes should run on specific NUMA nodes, ensuring that their memory allocations come from those nodes. This is particularly beneficial for performance tuning in multi-processor systems where memory access time varies based on the proximity of memory to the processor.
- When you set a NUMA policy for a command using numactl, the specified policy is inherited by all child processes spawned by that command. This cascading effect ensures that a consistent memory and scheduling policy is applied throughout a process hierarchy, which is crucial for maintaining performance optimizations in complex applications.
- Beyond just running processes, numactl can also set persistent policies for shared memory segments and files. This means you can configure memory policies for data multiple processes might access or for specific files requiring optimal memory placement.
In addition, numactl can set policies for SYSV shared memory segments or files in shmfs or hugetlbfs. These policies are persistent and affect all mappings from the shared memory or files. Options such as --huge, --offset, --strict, --mode, --length, --shmid, --shm, --file, --touch, and --dump allow for detailed control over memory allocation and policy enforcement for shared memory and files.
Node specifiers include all (all nodes), specific node numbers (number), ranges of nodes (number1-number2), and inverted selections (! nodes). These specifiers provide flexibility in defining which nodes are affected by the policies.
Table of Contents
Here is a comprehensive guide to the options available with the numactl command −
Syntax of numactl Command
The general syntax for the numactl command is as follows −
numactl [options] command
numactl Command Options
The following table provides a detailed breakdown of the different options available for the numactl command −
| Tag | Description |
|---|---|
| --interleave=nodes, -i nodes |
This option sets a memory interleave policy, where memory allocations are distributed in a round-robin manner across the specified nodes. If memory cannot be allocated on the current interleave target, it will fall back to other nodes. This approach helps balance memory usage across multiple nodes, improving performance for memory-intensive |
| --membind=nodes, -m nodes | With this option, memory allocations are restricted to the specified nodes. If there is insufficient memory on these nodes, the allocation will fail. |
| --cpunodebind=nodes, -N nodes | This option restricts process execution to the CPUs of the specified nodes. Nodes may consist of multiple CPUs, and this option ensures that the process runs only on the CPUs of the designated nodes, optimizing CPU usage and reducing cross-node traffic. |
| --physcpubind=cpus, -C cpus | This option restricts process execution to specific physical CPUs. The CPU numbers are based on the processor fields in /proc/cpuinfo. |
| --localalloc, -l | This option forces memory allocations to always occur on the current node. It ensures that memory usage is localized to the node where the process is running, which can reduce latency and improve performance. |
| --preferred=node | This option sets a preferred node for memory allocations. If memory cannot be allocated on the preferred node, it will fall back to other nodes. |
| --show, -s | This option displays the current NUMA policy settings for the process. It provides an overview of the memory and CPU binding policies in effect. |
| --hardware, -H | This option displays an inventory of available NUMA nodes on the system. It provides information about the system's NUMA topology, which is useful for understanding the hardware configuration. |
Examples of numactl Command in Linux
The following practical examples illustrate how numactl can manage process execution and memory allocation on NUMA systems, enhancing performance and optimizing resource utilization.
Show Current NUMA Policy Settings
To display the current NUMA policy settings for a process, run −
sudo numactl --show
This command provides an overview of the NUMA policies in effect for the process, helping you understand and optimize resource usage.
Check the Available NUMA Nodes on Your System
To check the available NUMA nodes on your system, you can use the following command −
sudo numactl --hardware
This command displays the available NUMA nodes and their corresponding CPUs.
Preferred Memory Allocation
To set node 0 as the preferred node for memory allocations and run a command, simply execute −
sudo numactl --preferred=0 ls
This will run the ls command with the preferred NUMA node set to 0.
Bind Specific Memory Range to a Node
To bind a specific 50MB range in a shared memory file to node 0, simply run −
sudo numactl --length=50M --offset=1M --file /dev/shm/myfile --membind=0 --touch
This command ensures that the specified memory range within the file is allocated on node 0, optimizing memory usage.
Interleave Shared Memory Across All Nodes
To interleave a SYSV shared memory region specified by a key file across all nodes, you can use the following command −
sudo numactl --length=50M --shm /tmp/shmkey --interleave=all
This command creates a 50 MB shared memory segment identified by the key file /tmp/shmkey and distributes the memory allocations across all available NUMA nodes.
Set Preferred Memory Node and Show State
To set the preferred memory node to 0 and display the resulting NUMA policy state, run the following command −
sudo numactl --preferred=0 numactl --show
This command sets node 0 as the preferred memory allocation node and immediately shows the current policy settings.
Interleave Memory on All CPUs
To run a large database with its memory interleaved on all CPUs, you can use the following command −
sudo numactl --interleave=all bigdatabase arguments
This command ensures that memory allocations are distributed in a round-robin manner across all available NUMA nodes, enhancing memory access performance for the database.
Conclusion
The numactl command offers versatile options and fine-grained control over process execution and memory allocation, making it an essential tool for optimizing performance in NUMA systems.
Whether you're managing high-performance computing tasks, database management, or any memory-intensive applications, numactl provides the necessary capabilities to tailor NUMA policies to your specific needs.