When SoftIRQs Pile Up
If you run top or htop on a busy Linux server, you will often see kernel threads named [ksoftirqd/0], [ksoftirqd/1], and so on — one per CPU core.
These threads exist to handle a specific problem: SoftIRQ overload.
Normally, SoftIRQs are processed inline — immediately after the Top Half returns, the kernel checks if any SoftIRQs are pending and runs them before returning to user space. This is fast and keeps latency low. But if SoftIRQs keep arriving faster than they can be processed (e.g., a flood of network traffic), the kernel would be stuck in SoftIRQ processing forever, starving user processes.
The kernel's solution: after processing SoftIRQs for a fixed time budget (or a fixed number of iterations), any remaining pending SoftIRQs are handed off to a dedicated per-core kernel thread — ksoftirqd/0 for Core 0, ksoftirqd/1 for Core 1, and so on, one per CPU. This thread runs at a normal scheduler priority, meaning user processes can preempt it.
The Danger Sign:
If ksoftirqd/N is consuming 80–100% CPU on a core, it is a clear signal that the system is overwhelmed by interrupt processing — almost always network traffic or storage I/O. The Bottom Half work is arriving faster than the CPU can drain it. Consequences include:
- Increased network latency — packets sit in the queue waiting to be processed.
- Packet drops — ring buffers overflow.
- User process starvation — applications get less CPU time.
Because ksoftirqd threads are ordinary (if specially named) processes, any standard process-monitoring tool shows their CPU consumption alongside everything else — a ksoftirqd/N thread pegged near 100% is as visible to top as a runaway application would be.
The fix is rarely "buy a faster CPU." It is usually: tune IRQ affinity to spread load across cores, enable RSS (Receive Side Scaling) on the NIC, or reduce interrupt rate via coalescing.