Why Do We Need Another Protocol?
VXLAN solved the scale problem, but it introduced a new constraint: its header is fixed and rigid. Once the standard was written, there was no room to add new fields without breaking compatibility. As cloud providers built increasingly sophisticated features — security group enforcement, distributed tracing, service mesh integration — they needed a way to carry metadata alongside the encapsulated packet.
GENEVE (Generic Network Virtualization Encapsulation), defined in RFC 8926, is the modern evolution. It does the same fundamental job as VXLAN — UDP encapsulation of Ethernet frames — but with one critical architectural difference: extensible, variable-length headers.
GENEVE uses a TLV (Type-Length-Value) options system in its header. Each option is a self-describing chunk of data:
- Type: What kind of metadata is this? (e.g., "Security Group ID", "Trace Context")
- Length: How many bytes does this option occupy?
- Value: The actual data.
A GENEVE packet with no options is nearly identical to VXLAN. But a GENEVE packet can carry arbitrary metadata that both endpoints understand, without breaking endpoints that don't recognize those options (they simply ignore unknown TLVs).
Real-world usage:
| Provider | Protocol | Why |
|---|---|---|
| AWS (Nitro hypervisor) | GENEVE | Carries security group policy IDs and flow metadata |
| VMware NSX-T | GENEVE | Carries context for distributed firewall enforcement |
| OVN (Open Virtual Network) | GENEVE | Carries logical port and policy metadata |
| Flannel (Kubernetes CNI) | VXLAN | Simpler use case, fixed headers sufficient |
GENEVE packet structure:
[ Outer Ethernet Header ]
[ Outer IP Header ]
[ Outer UDP Header ] ← Port 6081 (GENEVE uses a different port than VXLAN)
[ GENEVE Header ] ← Fixed base + variable TLV options
[ Inner Ethernet Frame ]
Note that GENEVE uses UDP port 6081, not 4789. This matters for firewall rules and packet capture filters.
A common interview question is: "What's the difference between VXLAN and GENEVE?" A weak answer is "GENEVE is newer." A strong answer is: "Both use UDP encapsulation of Ethernet frames, but VXLAN has a fixed 8-byte header with a 24-bit VNI, while GENEVE has an extensible header with TLV options that allow arbitrary metadata to be carried alongside the encapsulated frame. This is why AWS uses GENEVE on the Nitro hypervisor — they need to carry security group policy context in the packet itself so enforcement can happen at the destination VTEP without a separate policy lookup."
Key Point: GENEVE extends VXLAN's concept with extensible TLV headers, enabling cloud providers to embed metadata (security policies, trace IDs, service context) directly in the encapsulation layer without modifying the inner packet or requiring out-of-band lookups.