Fallback (Plan B)
When a Circuit Breaker opens, or a request fails after all retries are exhausted, what do you show the user? A raw "500 Internal Server Error" page destroys trust and provides no value.
A Fallback is a pre-defined alternative action that executes when the primary path fails. The goal is to degrade gracefully — to provide a reduced but still useful experience rather than a complete failure.
Common fallback strategies, in order of preference:
- Return cached data: If you fetched this data successfully 5 minutes ago and cached it, return the stale cache entry. Stale data is often better than no data.
- Return a default/static response: If the "Personalized Recommendations" service is down, return a hardcoded list of "Today's Popular Items." The user gets something useful.
- Return an empty/neutral response: If the "Discount Eligibility" service is down, assume the user is not eligible for a discount and proceed. The user may miss a discount, but the checkout flow is not broken.
- Fail visibly but partially: Hide the broken component from the UI (e.g., collapse the recommendations widget) while the rest of the page renders normally.
The worst fallback is no fallback — propagating the error all the way to the user as an unhandled exception.
Rate Limiting (The Bouncer)
Sometimes the problem is not that something is broken — it is that it is too popular. Rate Limiting restricts how many requests a client can make within a specific time window (e.g., 100 requests per minute per API key).
Server-Side Rate Limiting protects your API from being overwhelmed by a single aggressive client, a misconfigured retry loop, or a malicious bot. When a client exceeds the limit, you return 429 Too Many Requests with a Retry-After header indicating when they can try again. Common algorithms for implementing rate limiting include:
- Token Bucket: A bucket holds N tokens. Each request consumes one token. Tokens refill at a fixed rate. Allows short bursts up to bucket capacity.
- Leaky Bucket: Requests enter a queue and are processed at a fixed rate. Smooths out bursts but adds latency.
- Fixed Window Counter: Count requests per fixed time window (e.g., per minute). Simple but can allow 2x the limit at window boundaries.
- Sliding Window Log: Track timestamps of each request. More accurate but memory-intensive.
Client-Side Rate Limiting means your own application throttles itself before sending requests to a third-party API. Rather than sending requests as fast as possible and waiting to get a 429, you track your own usage and proactively slow down. This is especially important when using APIs that charge per request or that ban clients for repeated violations.
Interviewers at companies with large-scale APIs (Stripe, Twilio, AWS) often ask: "How would you implement rate limiting?" The expected answer covers the algorithm choice and its trade-offs. Token Bucket is the most common answer because it allows controlled bursting (useful for legitimate traffic spikes) while enforcing an average rate. Mention that in distributed systems, the counter must be stored in a shared, atomic store (like Redis with INCR and EXPIRE) rather than in-process memory, because multiple instances of your service will be handling requests from the same client.