Two Approaches
There are two ways to fix the encapsulation overhead problem. One is the right answer; one is a workaround you use when you can't do the right thing.
Option A: Jumbo Frames in the Underlay (Preferred)
Configure your physical switches, routers, and server NICs to support Jumbo Frames — an MTU of 9000 bytes (sometimes called 9216 to account for Ethernet overhead).
Underlay MTU: 9000 bytes
Overlay packet max size: 1550 bytes (1500 + 50 overhead)
Headroom remaining: 7450 bytes ← plenty of room
With jumbo frames, the physical network can carry fully-sized overlay packets with no fragmentation, no drops, and no performance penalty. This is the standard configuration in any well-designed data center. Enabling it means raising the MTU on every NIC and switch port along the path and making that setting persistent across reboots — a one-time physical/underlay configuration, not something each workload has to account for.
Option B: Shrink the Overlay MTU (Workaround)
If you cannot configure jumbo frames on the physical network (e.g., you're using a cloud provider's network where you don't control the underlay), configure the virtual interfaces inside your VMs or containers to use a smaller MTU:
VM interface MTU: 1450 bytes (reduced from 1500)
VXLAN overhead: + 50 bytes
Resulting packet: 1500 bytes ← fits in standard underlay MTU
In Kubernetes, CNI plugins like Flannel and Calico expose this MTU as a direct backend configuration option, so the pod network's effective MTU can be set explicitly rather than left at a default that doesn't account for VXLAN overhead.
Trade-offs:
| Approach | Performance | Complexity | When to Use |
|---|---|---|---|
| Jumbo Frames (underlay) | Best — no overhead penalty | Requires physical switch config | Own hardware, bare-metal, co-lo |
| Shrink overlay MTU | ~3% throughput reduction | Must configure every CNI/VM | Cloud VMs, managed networks |
A third option — TCP MSS Clamping:
For TCP traffic specifically, a firewall rule can rewrite the MSS (Maximum Segment Size) — the TCP-layer equivalent of MTU — announced during the handshake, clamping it down to a value that's guaranteed to fit through the overlay without fragmentation. This is commonly used in Kubernetes. This only helps TCP traffic. UDP-based applications (DNS, video streaming, QUIC) are unaffected by MSS clamping and still require proper MTU configuration.
Key Point: Jumbo frames in the underlay is the architecturally correct solution. Shrinking the overlay MTU is a valid workaround for environments where you don't control the physical network. MSS clamping is a useful band-aid for TCP but does not solve the underlying problem for all protocols.