How do you dig past surface-level answers to find those deep contributing factors? A simple but powerful technique is The 5 Whys, developed by Sakichi Toyoda and used extensively in Toyota's manufacturing system before being adopted by software engineering.
The method is straightforward: start with the immediate problem and ask "Why?" repeatedly. Each answer becomes the subject of the next "Why?" You stop when asking "why" no longer yields a useful, actionable answer. The name says "5" but in practice it might take 3 iterations or 7 — the number is a guideline, not a rule.
Example: The Website Crashed
- Why did the website crash? Because the database CPU spiked to 100% and stopped responding.
- Why did the database CPU spike? Because a new deployment introduced a query that performed a full table scan on a 50-million-row table.
- Why was a full table scan query deployed? Because it wasn't caught in code review or automated testing.
- Why wasn't it caught in testing? Because our staging environment has only 1,000 rows of test data, so the query ran in milliseconds there and appeared fine.
- Why doesn't staging have realistic data volumes? Because we have no automated process for sanitizing and copying production data to staging.
The Systemic Issue Identified: Staging environments with unrealistic data volumes create a false sense of security for performance-sensitive code.
Action Item: Build a data obfuscation and sampling pipeline that populates staging databases with a statistically representative subset of production data (anonymized for privacy compliance).
Where the 5 Whys Can Go Wrong
The 5 Whys is powerful but has failure modes worth knowing. First, different people asking "why" from the same starting point can arrive at completely different root issues — the technique is not deterministic. Second, it can be gamed: a team that wants to blame a person can steer the "whys" toward an individual rather than a system. Third, it works best for linear causal chains and can miss parallel contributing factors. For complex incidents, combine it with the Fishbone Diagram technique described next.
A common SRE interview question is: "Give me an example of a time you used the 5 Whys." The trap is answering with a chain that ends at a person ("because the engineer didn't test it"). A strong answer always ends at a systemic gap — a missing process, a tooling limitation, or an organizational ambiguity — and describes the concrete action item that resulted.