Mean Time To... Meaningless?
Executives love a single number that goes down over time. MTTR (Mean Time To Resolve) is the most common target. Leadership will ask for a chart showing MTTR improving quarter over quarter, and they will interpret a downward trend as proof that reliability is improving.
Be careful. In a meta-analysis, averages lie — especially when your incident distribution is highly skewed, which it almost always is.
Consider this scenario:
| Incident | Duration | Cause |
|---|---|---|
| A | 5 minutes | Typo in config, rolled back immediately |
| B | 5 minutes | Service restart cleared a deadlock |
| C | 24 hours | Data center network partition |
Average MTTR: ~8 hours.
That number is useless. It doesn't reflect the operational reality of incidents A and B (which were trivial), and it dramatically understates the severity of incident C. If you report "our MTTR improved from 10 hours to 8 hours," you might just be having fewer catastrophic outages — or you might be getting faster at rolling back trivial changes while your serious incidents get worse.
Use Percentiles, Not Averages
A more honest representation of MTTR uses percentiles:
- p50 (median) MTTR: The typical incident takes X minutes to resolve.
- p95 MTTR: 95% of incidents resolve within Y hours. The remaining 5% are your "long tail" — the serious ones that deserve dedicated attention.
- p99 MTTR: Your worst-case incidents. If this number is growing, you have a problem with your most severe outages regardless of what the average says.
Tracking p95 and p99 separately from the median gives you a much more honest picture of whether your reliability is actually improving.
Better Metric: SLO Burn Rate
Rather than focusing on resolution speed (which can incentivize quick, sloppy fixes that don't address root causes), the most strategically useful metric is SLO (Service Level Objective) Burn Rate.
- SLO: A target you've committed to, e.g., "99.9% of requests succeed" or "99.9% uptime per month."
- Error Budget: The allowable failure budget implied by the SLO. At 99.9% uptime, you have ~43 minutes of downtime per month to "spend."
- Burn Rate: How quickly you're consuming that budget. A burn rate of 1x means you'll exactly exhaust your budget by month end. A burn rate of 3x means you'll exhaust it in 10 days.
Tracking SLO burn rate across quarters aligns your meta-analysis with customer experience, not engineering speed. It answers the question that actually matters: "Are we keeping our promises to users?" A team that resolves incidents quickly but keeps having them may still be burning through its error budget faster than a team with slower MTTR but fewer incidents.
A common interview question for SRE roles is: "How do you know if your reliability is improving?" Answering with "MTTR is going down" is a red flag to experienced interviewers — it shows you haven't thought about the limitations of averages or the difference between speed and reliability. A strong answer references percentile-based MTTx metrics, SLO error budget burn rate trends, and the ratio of customer-detected to alert-detected incidents. These three together give a complete picture.