The Slow Drift Into Catastrophe
Imagine you move into a new apartment. The smoke detector chirps once on day one. You check — no fire, just a low battery. You plan to replace it. By day five, you've stopped hearing the chirp. By day thirty, a visitor asks "What's that beeping?" and you genuinely say "What beeping?" Your brain has reclassified a warning signal as background noise. Now apply this to a production system with 200 alerts.
This concept was formalized by sociologist Diane Vaughan while studying the 1986 NASA Challenger disaster. Engineers at Morton Thiokol had observed O-ring erosion on previous shuttle flights — a clear deviation from the design specification. Each time, the shuttle returned safely. Over successive flights, the erosion was reclassified from "anomaly" to "acceptable risk" to "expected behavior." On January 28, 1986, the O-rings failed catastrophically in cold weather, killing seven astronauts. The deviation had been normalized.
The Mechanism in Engineering
- Deviation occurs: A Redis cache timeout fires during peak traffic.
- No consequence follows: The application retries and the user never notices.
- Rationalization: The team labels it "Known Issue — intermittent Redis timeout under load."
- Acceptance: The alert threshold is raised, or the alert is muted, or the runbook says "Acknowledge and close."
- New baseline: The deviation is the normal. Safety margins have silently eroded.
- Catastrophe: A new code deployment increases Redis load by 15%. Combined with the "normal" timeouts, the cache layer collapses entirely. The application has no fallback because nobody built one — the timeouts were "normal," so why would you?
Real-World Engineering Examples
- Flaky tests: "Oh,
test_payment_flowfails randomly. Just re-run CI." One day it fails because of a real regression in payment logic. The team re-runs CI, the flaky test passes on retry, and the regression ships to production. - Alert fatigue: A team has 47 "warning" alerts that fire daily. Engineers stop reading them. A genuine warning about disk space at 90% gets lost in the noise. The disk fills. The database crashes.
- Manual workarounds: "After every deploy, SSH into the worker node and restart the queue consumer." This becomes tribal knowledge. A new team member deploys without the restart. Jobs pile up for hours before anyone notices.
The Fix — Process Patches
- The "Fix It or Delete It" Rule: If an alert fires and the team takes no action, the alert must either be fixed (resolve the underlying issue) or deleted (acknowledge it provides no value). A red dashboard that everyone ignores is worse than no dashboard — it trains the team to ignore red dashboards.
- Near-Miss Reviews: Treat incidents that almost happened with the same rigor as incidents that did happen. If a deploy nearly caused an outage but was caught by a canary, run a mini-post-mortem. The near-miss is data about your eroding safety margin.
- Periodic Baseline Audits: Every quarter, review your "known issues" list. For each item, ask: "If this appeared for the first time today, would we accept it?" If the answer is no, it shouldn't be accepted now either.
When asked "How do you handle alert fatigue?" in an SRE interview, don't just say "tune thresholds." Describe the normalization of deviance pattern by name. Explain that unactionable alerts actively degrade a team's ability to respond to real incidents because they train operators to ignore signals. Then describe the "fix it or delete it" policy. This demonstrates systems thinking, not just tooling knowledge.
Key Point: Normalization of deviance is the most dangerous bias on this list because it is collective, not individual. It's not one person making a bad call — it's an entire team gradually shifting its definition of "acceptable." This is why it requires process-level fixes (alert policies, review cadences, explicit standards) rather than individual vigilance. You cannot out-discipline a systemic drift.