The Problem with Running Databases in Deployments
Imagine trying to run a three-node MongoDB replica set using a standard Kubernetes Deployment. Here's what goes wrong:
- The three Pods start with random names:
mongo-xhk12,mongo-99sqa,mongo-ab3cd. - They need to form a cluster, but each node must know the stable DNS addresses of its peers. With random names, there are no stable addresses.
- The primary node crashes. Kubernetes replaces it with a new Pod that has a new random name and — critically — a new empty Persistent Volume Claim. The replica set now has a node with no data, and the other nodes don't recognize it.
The root cause: Deployments assume any Pod can replace any other Pod. Stateful applications break this assumption in three ways — they need stable names, stable storage, and ordered startup/shutdown.
What StatefulSets Provide
A StatefulSet is a controller designed for applications that require one or more of the following guarantees:
1. Stable, unique network identifiers
Pods get ordinal index names instead of random suffixes: web-0, web-1, web-2. These names are permanent — if web-1 is rescheduled to a different node, it keeps the name web-1. Combined with a Headless Service (a Service with clusterIP: None), each Pod gets a predictable DNS entry:
web-0.my-service.default.svc.cluster.local
web-1.my-service.default.svc.cluster.local
web-2.my-service.default.svc.cluster.local
Peers in a database cluster can hardcode these addresses in their configuration and trust they'll always resolve to the right node.
2. Stable, persistent storage
Each Pod gets its own Persistent Volume Claim (PVC) created from a volumeClaimTemplates definition. web-0 always gets data-web-0, web-1 always gets data-web-1. If web-1 is deleted and recreated, it reattaches to the same data-web-1 volume — its data is preserved.
3. Ordered, graceful deployment and scaling
StatefulSets are methodical. Pods are created, updated, and deleted one at a time in a defined order.
A Minimal StatefulSet Manifest
A StatefulSet spec looks like a Deployment spec with two additions: a serviceName field linking it to a Headless Service, and a volumeClaimTemplates section — a template for creating an individual PVC per Pod, rather than one shared volume for the whole set. The serviceName field links the StatefulSet to a Headless Service that provides the stable DNS entries. The volumeClaimTemplates section is what distinguishes StatefulSets from Deployments — it's a template for creating individual PVCs for each Pod.
Interviewers frequently ask: "What's the difference between a Deployment and a StatefulSet?" A weak answer lists features. A strong answer explains why those features exist: "Deployments assume Pod interchangeability. StatefulSets exist because distributed stateful systems like databases require stable identity — stable names for peer discovery, stable storage so data survives Pod rescheduling, and ordered startup so a primary is established before replicas try to connect to it."