Balancing the Load Across Cores
By default, on many systems, all hardware interrupts land on CPU Core 0. This is a severe bottleneck on modern servers with 32, 64, or 128 cores. Core 0 burns at 100% handling interrupts while the other 127 cores sit idle.
IRQ Pinning (also called IRQ Affinity) is the act of telling the APIC: "Route interrupts from this device to these specific CPU cores."
The kernel exposes this control through the /proc/irq/ filesystem. For each IRQ number, there is a directory containing:
| File | Purpose |
|---|---|
smp_affinity |
Bitmask (hex) of allowed CPUs |
smp_affinity_list |
Human-readable CPU list (e.g., 0-3) |
affinity_hint |
Driver's suggested affinity (read-only) |
Understanding the Bitmask:
The smp_affinity value is a hexadecimal bitmask where each bit represents one CPU core. Bit 0 = Core 0, Bit 1 = Core 1, and so on.
| Hex value | Binary | Meaning |
|---|---|---|
1 |
0001 |
Core 0 only |
2 |
0010 |
Core 1 only |
3 |
0011 |
Core 0 and Core 1 |
f |
1111 |
Cores 0, 1, 2, and 3 |
ff |
11111111 |
Cores 0 through 7 |
Pinning an IRQ to a specific core means writing the target bitmask (or the equivalent human-readable core list) into that IRQ's smp_affinity file — once an engineer identifies which IRQ number belongs to the device in question, redirecting it is a single write to a pseudo-file, which is why this is a runtime operation rather than something requiring a reboot or driver reload.
Why this matters for multi-queue NICs:
Modern high-speed NICs support Multiple Queues — the NIC has multiple receive (RX) and transmit (TX) queues, each with its own IRQ. The ideal configuration pins each queue's IRQ to a different CPU core, so packet processing is parallelized across all cores instead of funneling through one. This is called RSS (Receive Side Scaling) when done in hardware, and RPS (Receive Packet Steering) when done in software.
A common SRE interview question is: "Your server has 32 cores but network throughput plateaus and Core 0 is at 100% while others are idle. What do you investigate?" The answer is IRQ affinity — all NIC interrupts are landing on Core 0. The fix is to check /proc/irq/*/smp_affinity for the NIC's IRQs, and either pin them to specific cores manually or enable irqbalance (a daemon that does this automatically). Follow-up: "When would you NOT use irqbalance?" — Answer: latency-sensitive workloads where you want deterministic, static pinning rather than dynamic rebalancing.
irqbalance — The Automatic Alternative:
For general-purpose servers, the irqbalance daemon automatically distributes IRQs across cores based on CPU load. It is a reasonable default. For latency-sensitive workloads (databases, trading systems), disable irqbalance and pin IRQs manually — dynamic rebalancing introduces jitter, since the daemon can move an IRQ to a different core mid-workload in response to shifting load, which is exactly the kind of non-determinism a latency-sensitive service wants to avoid.