Introduction: The Kernel Bottleneck Revisited

The Cost of Being "General Purpose"

Think of the Linux kernel's networking stack like a full-service post office. Every package that arrives — regardless of whether it's a birthday card or a critical legal document — gets logged, weighed, sorted, stamped, and routed through the same multi-step process. That thoroughness is exactly what makes it reliable. But if you need to process ten million packages per hour, the overhead of that process becomes the bottleneck, not the volume of packages.

The Linux kernel is a miracle of engineering. It runs on everything from supercomputers to embedded routers. To achieve this universality, its networking stack handles everything: TCP, UDP, routing, firewalls, NAT, ARP, statistics, logging, and more. Every packet entering a Linux server is allocated a data structure called an sk_buff (socket buffer) — a rich, general-purpose object that carries the packet and all its metadata through the entire kernel networking pipeline. This structure can be dozens of layers deep: the packet passes through the NIC driver, the traffic control (tc) subsystem, netfilter hooks (iptables), routing tables, and finally into a socket buffer where a user-space application can read it.

• Pros: Feature-rich, standards-compliant, easy to use with familiar tools (tcpdump, iptables, ss).
• Cons: Slow at extreme scale. Allocating an sk_buff and traversing 50+ layers of kernel code costs CPU cycles — cycles that add up catastrophically at high packet rates.

When you need to handle 100 million packets per second (as Cloudflare, Facebook, and major CDNs do), the overhead of sk_buff allocation alone saturates CPUs before the kernel even inspects the packet payload. A single sk_buff allocation involves memory initialization, reference counting, and cache pressure. At 100 Mpps, that's 100 million allocations per second — each one a small tax that collectively becomes ruinous.