The Modulo Problem
Imagine you are building a distributed cache with 4 servers — think of it like 4 lockers in a school hallway, and you need a rule to decide which locker holds which student's belongings. The standard rule is "Modulo Hashing": take a numeric fingerprint (hash) of the key, divide by the number of servers, and use the remainder as the server index.
server_index = hash("user_123") % 4
If hash("user_123") produces 10, then 10 % 4 = 2. The data goes to Server 2. Simple, fast, deterministic.
The Disaster
What happens when you add a 5th server to handle more load? The formula changes:
server_index = hash("user_123") % 5
10 % 5 = 0 ← now points to Server 0, not Server 2
Changing the divisor from 4 to 5 invalidates the mapping for almost every single key. Mathematically, when you change the modulus from N to N+1, approximately N/(N+1) of all keys map to a different server — that's ~80% of your cache gone in an instant.
- Immediate result: Millions of cache misses fire simultaneously.
- Downstream result: Every miss falls through to your database, which was sized to handle cache-assisted load, not raw traffic. The database saturates and your service goes down.
- Recovery: Even after the cache warms back up, the thundering herd of requests during the warm-up window can cause cascading failures across dependent services.
This is not a theoretical concern. It is the exact failure mode that has caused production outages at companies scaling their first distributed cache. We need a scheme where adding or removing a server moves only a small, bounded fraction of keys. That scheme is Consistent Hashing.