The Magic of the "Service" IP
Think of a Kubernetes Service IP like a phone number for a department at a large company. When you call the main number, you don't know which employee will pick up — the switchboard routes you to whoever is available. The number itself doesn't belong to any one desk; it exists only in the switchboard's routing logic. Kubernetes Services work the same way: the ClusterIP is a virtual number that exists only in the kernel's routing rules, not on any physical network interface.
If you SSH into a node and run ip addr, you won't see the ClusterIP listed on any interface. It has no MAC address, no physical home. It exists purely as a rule: "when you see traffic destined for this IP, redirect it here." The component responsible for writing those rules is kube-proxy. It runs as a DaemonSet on every node in the cluster, watches the Kubernetes API for Service and Endpoint changes, and translates those abstract objects into concrete kernel-level routing instructions.
Understanding this mechanism is essential for debugging networking failures. When a Service stops working, the problem is almost always in one of three places: the kube-proxy rules are missing or wrong, the CNI plugin failed to set up Pod networking, or the underlying node-to-node connectivity is broken. This lesson walks through all three layers from first principles.