The Ship Analogy
Modern ships are built with watertight compartments called bulkheads. If the hull is breached and one compartment floods, the water is contained there. The ship may list or slow down, but it will not sink. Without bulkheads, a single breach anywhere in the hull would flood the entire vessel.
The Titanic famously had bulkheads, but they did not extend high enough — water could spill over the top from one compartment to the next. The lesson: bulkheads only work if the isolation is complete.
Thread Pool Isolation
In software, if your service calls both Service A and Service B, you do not want a slowdown in Service A to make Service B unavailable.
Consider a single, shared thread pool with 100 threads handling all outgoing requests. Service A becomes slow — each request takes 30 seconds instead of 50ms. Incoming requests to your service keep arriving. Each one grabs a thread and waits for Service A. Within seconds, all 100 threads are stuck waiting for Service A. Now a request comes in that only needs to call Service B — the healthy, fast service. There are no threads available to handle it. Service B is effectively down, even though it is perfectly healthy, because Service A consumed all shared resources.
The Bulkhead Pattern isolates resources by dependency. You assign a dedicated thread pool (or connection pool, or semaphore) for calls to Service A, and a completely separate pool for Service B. If Service A's pool fills up, only calls to Service A are affected. Service B's pool remains untouched and fully functional.
Semaphore vs. Thread Pool Bulkheads
There are two common implementations:
- Thread Pool Bulkhead: Each dependency gets its own pool of threads. Requests to that dependency run on those threads. If the pool is full, new requests are rejected immediately. This provides strong isolation but has overhead from thread management.
- Semaphore Bulkhead: A semaphore (a counter) limits the number of concurrent calls to a dependency. If the semaphore count is exhausted, new requests are rejected. This is lighter weight but does not protect against thread exhaustion if calls are synchronous and blocking.
Key Point: Bulkheads are often the most overlooked resiliency pattern because their benefit is invisible until a dependency fails. When Service A is healthy, having separate thread pools feels like wasted overhead. When Service A degrades, bulkheads are the only thing preventing a cascading failure that takes down your entire service. Size your bulkhead pools based on the expected concurrency and latency of each dependency, not as a uniform fraction of your total thread capacity.