# NUMA on Dual-CPU Servers: Process and Memory Pinning

> NUMA on a dual-processor server affects memory latency: we cover node topology, binding processes with numactl, and common database configuration mistakes.

## What NUMA is and why it matters on two processors

NUMA (Non-Uniform Memory Access) is the memory architecture of a two-socket server, where each CPU has "local" memory with fast access and "remote" memory on the other socket, reached through the inter-processor bus. The topic matters most for servers with two or more sockets — how the [processor was chosen](https://www.zevshost.net/en/kb/vybor-protsessora-servera/) directly determines the number of NUMA nodes and their size.

If an application is not NUMA-aware, the OS scheduler can place a process on one socket while its memory sits on another. The result is a performance drop with no obvious cause in CPU monitoring.

## How to check NUMA topology

```
lscpu | grep -i numa
numactl --hardware
```

The numactl --hardware command shows the number of nodes, the memory size on each, and the node distances. A value of 10 means access to the local node, 20-21 means access to a neighboring one: the higher the number, the more expensive a miss is.

## The problem: memory on the "wrong" node

A typical scenario: a two-processor server with 256 GB of memory, an application started without any binding. The Linux scheduler moves the process between cores on different sockets while its allocated memory stays on the first node. Every access to the second socket goes through the inter-processor interconnect, and latency grows.

This is especially visible on [database servers](https://www.zevshost.net/en/kb/server-pod-bazu-dannyh/), where cache and buffers need to live next to the cores that read them thousands of times per second.

## Binding processes: numactl and cpuset

Explicit binding of a process to a node solves the problem directly:

```
numactl --cpunodebind=0 --membind=0 /usr/bin/mysqld
numactl --interleave=all /usr/bin/myapp
```

The --cpunodebind flag pins the process to node 0's cores, --membind pins it to that same node's memory. The --interleave=all flag spreads memory evenly across all nodes — useful for applications that are not NUMA-aware and gain nothing from locality.

| Parameter | What it does | When to use it |
| --- | --- | --- |
| --cpunodebind | Pins the node's cores | Databases, JVM with a large heap |
| --membind | Pins the node's memory | Together with cpunodebind |
| --interleave=all | Interleaves memory across nodes | NUMA-unaware applications |
| --preferred | Prefers a node but does not forbid others | Flexible workloads |

## NUMA in databases and virtualization

PostgreSQL and MySQL recommend running one instance per NUMA node for large memory sizes, or explicitly binding the process with numactl if there is a single instance for the whole server. In virtualization (KVM, Proxmox) you should allocate a virtual machine's vCPUs and memory within a single physical node — otherwise the hypervisor runs into the same cross-node traffic problem.

Before changing any binding, measure the effect: run a test before and after with [sysbench](https://www.zevshost.net/en/kb/benchmark-servera-fio-sysbench/) under a load close to the real one, not a synthetic single-threaded run.

## NUMA tuning checklist

- Check the topology with numactl --hardware before tuning, not after.
- Bind the process to CPU and memory together, not separately.
- For NUMA-unaware applications use --interleave=all instead of manual binding.
- Measure latency and throughput before and after — the gain is not always obvious under light load.

## Common NUMA-Related Errors

```
numactl: Warning: cpu binding outside of valid numa node cpuset

libnuma: Warning: node argument 4 is out of range

mbind: Cannot allocate memory

numactl: command not found
```

| Message | Cause | Fix |
| --- | --- | --- |
| cpu binding outside of valid numa node cpuset | The specified cores don't belong to the node given in `--membind` | Check the node's core numbers with `numactl --hardware` before composing the command |
| node argument N is out of range | The command specifies a node number that doesn't physically exist on the server (for example, node 4 on a dual-socket machine with nodes 0 and 1) | Check the actual node count: `numactl --hardware \| grep available` |
| mbind: Cannot allocate memory | The requested NUMA node physically doesn't have enough free memory for a hard `--membind` | Use `--preferred` instead of `--membind` to allow allocation on another node when memory is short |
| numactl: command not found | The numactl package isn't installed — it's not part of the minimal install on most distributions | Install the package: `apt install numactl` or `dnf install numactl` |

## Ways to Manage NUMA Binding: A Comparison

| Method | Scope | When to use it |
| --- | --- | --- |
| numactl (for one process) | A single process being launched | Quickly testing a hypothesis and manually starting a database or service |
| cgroups cpuset | A process group, a systemd unit, a container | A persistent binding for a service that gets restarted by the system, not by hand |
| BIOS Node Interleaving | The entire system | Temporary diagnostics only — disables NUMA entirely, not for production |

## When NUMA Binding Has No Effect

- The server is single-socket — the NUMA topology collapses to one node, and `numactl --hardware` will only show a node distance of 10; there's nothing to bind
- The application's working set is smaller than one NUMA node's memory — the scheduler already keeps the process and its memory together almost all the time without manual intervention
- The workload is heavily multithreaded and uses every core on the server evenly — hard binding to one node reduces parallelism here instead of speeding it up
- The virtual machine already migrates between cluster nodes (live migration) — statically pinning vCPUs to a physical server's NUMA node loses its point once the VM moves to another machine

## Verifying the Result

```
# Local and remote memory accesses by node
numastat -p $(pgrep -f mysqld)

# Memory access latency before and after binding
numactl --cpunodebind=0 --membind=0 -- perf stat -e task-clock,cycles ./benchmark

# Final memory distribution of the process across nodes
cat /proc/$(pgrep -f mysqld)/numa_maps | head -5
```

1. Run `numastat -p PID` before and after binding — the share of **numa\_foreign** and **other\_node** should drop noticeably
2. Measure the target load metric (requests per second, response time) both ways — without binding and with it — on the same traffic profile
3. Check via `/proc/PID/numa_maps` that most of the process's pages actually sit on the expected node rather than being spread evenly
4. Repeat the measurement after a day of real load — a synthetic test right after startup doesn't always show the same behavior as a warmed-up cache under steady state

Setting up virtualization itself, where a physical server's NUMA topology is passed through to guest machines, is covered in [virtualization on a dedicated server](https://www.zevshost.net/en/kb/virtualizatsiya-na-vydelennom-servere/).

---

Source: https://www.zevshost.net/en/kb/numa-privyazka-processov/
Updated: 2026-09-30