Why Timeouts Are Non-Negotiable
Every network call in a distributed system must have a timeout. This sounds obvious, but it is one of the most commonly missed configurations in production systems.
Without a timeout, a call to a slow or unresponsive service will block the calling thread indefinitely. The thread is not doing useful work — it is just waiting. As more requests arrive, more threads get stuck waiting. Eventually, the thread pool is exhausted and your service becomes unresponsive, even though the problem originated in a completely different service. This is cascading failure, and missing timeouts are one of its most common root causes.
Setting Timeout Values
Choosing the right timeout value requires understanding your system's latency profile:
- Too short: You time out on requests that would have succeeded, causing unnecessary failures and retries.
- Too long: You hold threads for too long during a downstream outage, exhausting resources before the circuit breaker can trip.
A practical approach: measure the P99 latency (the 99th percentile response time) of the downstream service under normal conditions. Set your timeout to 2–3x that value. This gives legitimate slow requests room to complete while still cutting off truly stuck calls quickly.
Timeout Budgets and Deadline Propagation
In a chain of microservices (A calls B calls C calls D), each service adds its own timeout. If A gives itself 5 seconds total, and B takes 3 seconds, B only has 2 seconds to call C. If B sets its own independent 5-second timeout for calling C, the math breaks down — B might wait 5 seconds for C even though A has already given up and returned an error to the user.
Deadline propagation solves this by passing the remaining time budget through the call chain, typically as a request header (e.g., grpc-timeout in gRPC, or a custom X-Request-Deadline header in REST). Each service checks the remaining budget before making downstream calls and aborts early if there is not enough time left to be useful.
Key Point: Timeouts, circuit breakers, and bulkheads form a defense-in-depth strategy against slow dependencies. Timeouts are the first line of defense (cut off individual slow calls). Bulkheads are the second (limit how many threads can be stuck at once). Circuit breakers are the third (stop sending traffic entirely when a dependency is systemically slow). All three are needed — each covers failure modes the others miss.