Why Rate Limit?
Every system has a breaking point. Whether it's a database that can only handle 1,000 writes per second or an API designed for fair usage across thousands of clients, uncontrolled traffic is a liability. A single misbehaving client — or a sudden surge of legitimate users — can exhaust resources and cause cascading failures across your entire system.
Rate Limiting is the practice of rejecting or delaying requests that exceed a defined threshold. It is a first-line defense for availability, fairness, and cost control. Without it, your system's reliability is only as good as your least-predictable client.
Rate limiting is not a single algorithm — it's a family of strategies. The simplest is the Fixed Window Counter: count requests in a time window (e.g., 100 requests per minute), reset the counter when the window expires. It's easy to implement but has a critical flaw: a client can send 100 requests at 12:00:59 and another 100 at 12:01:01, effectively sending 200 requests in 2 seconds while technically obeying the limit. The Sliding Log algorithm fixes this by tracking exact timestamps of every request, but it's memory-intensive at scale.
The two most foundational and widely deployed algorithms — used in network hardware, cloud APIs, and distributed systems — are the Leaky Bucket and the Token Bucket. They sound similar, but they behave very differently when traffic arrives in bursts. Understanding the distinction is essential for designing systems that are both resilient and user-friendly.