Free — no signup required

Canary Deployments: Testing with Real Traffic

2 min read

The Canary in the Coal Mine

Before modern gas detectors, coal miners carried canaries into mines. If toxic gas was present, the canary would be affected first — giving miners an early warning to evacuate. A canary deployment uses the same principle: expose a small portion of real production traffic to the new version first, and use that as your early warning system.

A Canary Deployment runs two versions of your application simultaneously in production:

  • Stable (Baseline): The current, known-good version (V1). Receives the majority of traffic.
  • Canary: The new version (V2). Receives a small slice of traffic — typically 1–10%.

Real users hit the canary, generating real metrics under real load conditions. If V2 has a bug that only manifests with certain data patterns or at production scale, the canary will reveal it while limiting the blast radius to only the users on that small traffic slice.

How Traffic Splitting Works

Traffic splitting is typically implemented at the infrastructure layer, not the application layer. Common mechanisms include: a Kubernetes-native progressive delivery controller like Argo Rollouts, which steps a canary's traffic weight upward in stages (e.g. 5%, then 25%, then 50%) with a pause between each step to observe results before promoting to 100%; a weighted upstream configuration at the ingress layer, splitting traffic between the stable and canary backend by percentage; or a service mesh routing rule (like an Istio VirtualService) that achieves the same weighted split at the mesh layer instead of the ingress. All three approaches express the same underlying idea — a percentage split between two versions — just enforced at a different point in the traffic path.

The key insight is that traffic splitting happens before the request reaches your application code. The application itself doesn't know it's a canary — it just receives requests and responds normally.

This is one of 18 chapters

Every chapter is free — Kubernetes, Terraform, SRE, distributed systems, and more — with fast daily review built in.

Sign up free