Fast vs. Slow Thinking in Outages
When a production system fails, your brain operates in one of two modes. Understanding these modes is the single most important meta-skill in incident response—it determines whether you solve the problem or just make it quieter for a while.
- Pattern Matching (System 1): This is your brain's fast, intuitive autopilot. It sees "High CPU" and immediately screams "Infinite Loop!" because that's what happened last week. It operates on correlation—two things happened together before, so they must be related now.
- Causal Reasoning (System 2): This is slow, deliberate, and mentally expensive. It asks, "What specifically caused the CPU to rise? Was it user traffic? A background task? A garbage collection cycle? A change in the JVM's heap configuration?" It seeks causation—a provable chain of events where one thing directly produces another.
These terms (System 1 and System 2) come from psychologist Daniel Kahneman's research on how humans make decisions. System 1 evolved to keep you alive—"that shape looks like a tiger, run!" System 2 evolved to solve novel problems—"how do I build a trap for the tiger?" In incident response, System 1 gets you moving fast but often in the wrong direction. System 2 gets you to the truth but feels painfully slow when a pager is screaming.
The critical insight is that both modes are necessary. System 1 is how you notice that something is wrong in the first place—pattern recognition fires before you've consciously processed the dashboard. System 2 is how you figure out what to do about it. The failure mode isn't using System 1; it's stopping there.
The Danger of Shotgun Debugging
In complex distributed systems, pattern matching alone is the enemy of truth. It leads to "shotgun debugging"—randomly restarting services, reverting code, or scaling up infrastructure because "it feels right." This approach often masks the real root cause, which then resurfaces days or weeks later, usually at 3 AM and usually worse.
The term "shotgun debugging" comes from the image of firing a shotgun at a problem: you spray a wide blast of changes and hope one of them hits. The cost is enormous. You've now changed multiple variables simultaneously, so even if the problem goes away, you don't know which change fixed it—or whether it fixed itself for an unrelated reason (like a cron job finishing).
Consider this scenario: Your API response time jumps from 50ms to 2 seconds. The pattern-matching brain says:
- "We deployed yesterday → rollback!"
- "Memory looks high → restart all pods!"
- "It's probably the database → add read replicas!"
The causal-reasoning brain asks:
- "Which specific endpoints are slow? All of them, or just
/api/search?" - "Did the slowness start before or after the deployment? What's the exact timestamp of each?"
- "What changed in the request pattern at that exact moment? Did traffic spike? Did a new client start hitting us?"
- "Are downstream dependencies healthy? What do the traces show?"
The first approach might accidentally fix the problem. The second approach will definitely find the problem, and more importantly, will prevent it from happening again.
The hidden cost of shotgun debugging compounds over time. Each masked incident leaves a landmine in the system—an unresolved root cause that will detonate again under slightly different conditions. Teams that shotgun debug accumulate technical debt not just in their code, but in their understanding of their own systems.
When asked "How would you debug X?", interviewers are testing whether you shotgun debug or reason systematically. Structure your answer as: (1) "First, I'd observe—what do metrics, logs, and traces tell me about the scope and shape of the problem?" (2) "Then I'd hypothesize—based on the evidence, here's what I think is happening and why." (3) "Then I'd test—here's the specific action I'd take to confirm or reject that hypothesis before touching anything." Articulating your hypothesis before describing your action is what separates senior engineers from junior ones in interviews. Interviewers specifically listen for whether you say "I would check X to confirm" versus "I would just restart the service."
Key Point: Pattern matching isn't inherently bad—it's essential for rapid triage. The danger is when you stop at pattern matching. Senior engineers use System 1 to generate hypotheses quickly, then immediately switch to System 2 to validate them. The sequence is: intuition generates candidates, evidence selects the winner. A useful mental model is to treat every System 1 hypothesis as a question, not an answer: "Could this be an infinite loop?" rather than "This is an infinite loop."