Stop Calling the Dead Service
Retries with backoff handle transient failures — brief hiccups that resolve quickly. But what if a service is completely down for five minutes? Retrying every few seconds is pointless: you are wasting CPU cycles, consuming thread pool capacity, and making your users wait for timeouts that will never succeed.
A Circuit Breaker is a stateful wrapper around a risky operation. It monitors the health of the operation and, when failures exceed a threshold, it "trips" — stopping all calls to the failing service instantly. The name comes directly from the electrical circuit breaker in your home's fuse box: when a dangerous current surge occurs, the breaker trips and cuts the circuit before the wiring catches fire.
The Three States
1. CLOSED (Normal Operation)
Traffic flows through normally. The circuit breaker counts failures in a rolling time window (e.g., "5 failures in the last 10 seconds"). Below the threshold, every request goes through. When failures exceed the threshold, the breaker trips to OPEN.
2. OPEN (Fail Fast)
All requests are rejected immediately without attempting to contact the remote service. The circuit breaker throws an error (or returns a cached fallback response) in microseconds. This has two benefits: users get a fast, predictable error instead of a 60-second TCP timeout, and the failing service gets zero traffic — giving it space to recover.
3. HALF-OPEN (Probing)
After a configured timeout (e.g., 30 seconds), the breaker transitions to HALF-OPEN and allows exactly one request through as a probe:
- If the probe succeeds: the service has recovered. Reset to CLOSED.
- If the probe fails: the service is still down. Trip back to OPEN and wait another 30 seconds.
Fallback Responses
A circuit breaker in the OPEN state does not have to just throw an error. Well-designed systems return a fallback: a cached result, a default value, or a degraded-but-functional response. For example, a product recommendation service that is down might return a static list of bestsellers instead of personalized recommendations. The user experience degrades gracefully rather than breaking entirely.
Failure Counting Strategies
Simple count-based thresholds ("5 failures") can be noisy. Production circuit breakers often use a sliding window — either count-based (last N requests) or time-based (last N seconds) — and trip on a failure rate (e.g., "more than 50% of the last 20 requests failed") rather than a raw count. This prevents a single burst of traffic from tripping the breaker unfairly.
A common interview question is: "What is the difference between a Circuit Breaker and a Retry?" They solve different problems and work at different timescales. Retries handle transient failures (the service is briefly unavailable but will recover in milliseconds). Circuit Breakers handle sustained failures (the service is down for seconds or minutes). In practice, you use both: retries for the first few attempts, and a circuit breaker to stop retrying once the failure rate indicates a real outage. The circuit breaker sits outside the retry loop.
Key Point — The Circuit Breaker Is a State Machine
The critical insight is that a circuit breaker maintains state across requests. This means it must be a shared, singleton object — not instantiated per-request. If you create a new circuit breaker for every HTTP call, it has no memory of previous failures and can never trip. In multi-threaded or multi-process environments, state management becomes a coordination problem: either use a thread-safe in-process object (for single-process apps) or an external store like Redis (for distributed systems where multiple application instances share the same circuit breaker state).