Free — no signup required

Production Readiness Reviews (PRR)

4 min read

The SRE Contract

In most organizations, when a new service is built, it gets handed to operations with a shrug and a prayer. In mature engineering organizations — Google, LinkedIn, Stripe — this is not allowed. You cannot simply throw code over the wall to the SRE team and expect them to carry the pager for something they have never seen.

Instead, you must pass a Production Readiness Review (PRR). A PRR is a structured audit where the SRE team evaluates a new service against a defined set of reliability criteria. The outcome is binary: if the service passes, SREs agree to support it (they carry the pager). If it fails, the development team keeps the pager until the gaps are fixed. This creates a powerful incentive: developers who want to sleep through the night are motivated to build reliable systems.

What a PRR Actually Examines

A PRR is not a rubber-stamp exercise or a pile of paperwork. It is a structured conversation between the team that built the service and the team that will operate it. The checklist forces both sides to think through failure modes before they happen in production.

1. Architecture & Failure Modes
- Is there a Single Point of Failure (SPOF)? A SPOF is any component whose failure brings down the entire service — a single database, a single availability zone, a single external dependency with no fallback.
- Are there retry storms? If Service A retries failed calls to Service B aggressively, and Service B is degraded, Service A can amplify the problem and take down Service B entirely.
- Is there a circuit breaker? A circuit breaker stops sending requests to a failing dependency, giving it time to recover instead of hammering it with retries.

2. Observability
- Are the Four Golden Signals instrumented? These are: Latency (how long requests take), Traffic (how many requests per second), Errors (what fraction of requests fail), and Saturation (how full is the system — CPU, memory, queue depth).
- Are logs structured (JSON, key-value pairs) rather than free-form text? Structured logs can be queried programmatically. Free-form logs require humans to grep through them at 2 AM.
- Is there distributed tracing? For microservices, you need to follow a single request across multiple services to diagnose latency.

3. Alerting
- Are alerts actionable? An alert that fires but requires no human action is noise. Every alert should have a clear "what to do" response.
- Do alerts go to a defined on-call rotation, not just a shared Slack channel that everyone ignores?
- Is there a defined SLO (Service Level Objective) — e.g., "99.9% of requests complete in under 200ms" — and does the alerting enforce it?

4. Runbooks and Playbooks
- Is there a written guide for "What to do if this breaks"? A runbook is a step-by-step procedure for handling a known failure mode. Without it, every incident is a first-time experience, even if the same failure has happened ten times before.
- Are runbooks linked directly from alerts? When an alert fires at 3 AM, the on-call engineer should be one click away from the relevant runbook.

5. Release Engineering
- Is the build reproducible? Can you build the exact same artifact from source control at any time?
- Can you roll back in under 5 minutes? Rollback is your most important safety net. If it requires a manual process, a database migration, or a phone call to a specific engineer, it is not a real rollback.
- Is there a canary or staged rollout process? Deploying to 1% of traffic before 100% catches bugs before they affect everyone.

6. Dependency Analysis
- What external services does this system call? What happens to your service if each of those dependencies goes down? The answer should never be "our service also goes down."
- Are there hard dependencies (service cannot function without them) and soft dependencies (service degrades gracefully without them)? Hard dependencies should be minimized.

The "Gate" Concept

The PRR acts as a quality gate — a checkpoint that a service must pass before it is allowed into production. This gate is not bureaucracy for its own sake. It forces development teams to prioritize reliability features (circuit breakers, structured logging, runbooks) that they would otherwise defer indefinitely in favor of shipping new features. The PRR makes reliability a first-class requirement, not an afterthought.

Interview Tip

Interviewers at companies with SRE practices frequently ask: "How do you decide whether a service is ready for production?" The weak answer is "when QA signs off." The strong answer describes a PRR process: "We evaluate against a checklist covering architecture (no SPOFs, circuit breakers), observability (Golden Signals instrumented, structured logs), alerting (actionable alerts tied to SLOs), runbooks, and rollback capability. The SRE team only accepts pager responsibility once all critical items pass." This demonstrates you understand that production readiness is a multi-dimensional property, not just functional correctness.

Key Point: The PRR is most valuable as a conversation, not a form. The checklist is a prompt for discussion between the people who built the system and the people who will operate it at 3 AM. The goal is shared understanding of how the system fails and what to do about it — not a completed PDF filed in a wiki nobody reads.

This is one of 18 chapters

Every chapter is free — Kubernetes, Terraform, SRE, distributed systems, and more — with fast daily review built in.

Sign up free