The Entry and Exit Points of the Overlay
The encapsulation and decapsulation work described above has to happen somewhere. That "somewhere" is called a VTEP (VXLAN Tunnel Endpoint). Every server participating in a VXLAN overlay has at least one VTEP — it is the boundary agent that translates between the virtual world and the physical world.
A VTEP has two identities simultaneously:
- A physical IP address on the underlay network (e.g., 192.168.1.10) — this is how other VTEPs find it.
- A virtual MAC address on the overlay network — this is what VMs see.
VTEPs can be implemented in two ways:
- Software VTEP: Running inside the hypervisor (KVM, VMware ESXi) or a virtual switch like Open vSwitch (OVS). This is how AWS, GCP, and Azure implement overlays — no special hardware required. The CPU on each server handles the encapsulation.
- Hardware VTEP: A physical Top-of-Rack switch with an ASIC that performs VXLAN encapsulation in hardware at line rate. Used in high-performance environments where software encapsulation would consume too many CPU cycles.
The complete packet flow:
VM A (10.0.0.5)
│ sends Ethernet frame to VM B (10.0.0.6)
▼
Source VTEP (on Server 1, physical IP 192.168.1.10)
│ looks up: "VM B is on Server 2 at 192.168.1.20"
│ wraps frame in VXLAN UDP packet
│ sends to 192.168.1.20
▼
Physical Network (Underlay)
│ routes UDP packet from 192.168.1.10 → 192.168.1.20
▼
Destination VTEP (on Server 2, physical IP 192.168.1.20)
│ receives UDP packet on port 4789
│ strips VXLAN headers
│ delivers original Ethernet frame
▼
VM B (10.0.0.6)
│ receives the frame as if VM A was on the same switch
How does the source VTEP know where VM B lives? This is the VTEP learning problem. There are two approaches:
- Flood and Learn (traditional): The VTEP broadcasts to all VTEPs and learns the mapping from replies — similar to how a physical switch learns MAC addresses. Simple but generates a lot of broadcast traffic.
- Control Plane Distribution (modern): A central controller (like BGP EVPN, or a Kubernetes CNI controller) pre-populates each VTEP with a mapping table. No flooding required. This is how production cloud environments work.
Interviewers frequently ask: "How does a VTEP know which physical server a VM is on?" The answer distinguishes candidates who understand overlay networking deeply. The key insight is that VXLAN itself is just the data plane — it defines how packets are encapsulated. The control plane (BGP EVPN, a CNI plugin, or a proprietary SDN controller) is a separate concern that distributes the MAC-to-VTEP mapping table. A common follow-up is: "What happens if the control plane goes down?" — answer: existing flows continue (the data plane still works), but new VM migrations or new VM-to-VM connections cannot be established until the control plane recovers.
Key Point: VTEPs are the boundary between the virtual and physical worlds. The data plane (VXLAN encapsulation) and the control plane (how VTEPs learn about each other) are separate concerns — understanding both is essential for production troubleshooting.