Side-by-Side
| Feature | Leaky Bucket | Token Bucket |
|---|---|---|
| Output Rate | Constant (always fixed) | Variable (can burst up to capacity) |
| Burst Capability | None — bursts are smoothed or dropped | Yes — up to the bucket's token capacity |
| Implementation | FIFO queue + fixed-rate dequeuer | Integer counter + refill rate |
| Space per Client | O(queue size) — stores pending requests | O(1) — stores only two numbers |
| Primary Goal | Protect the downstream service from shock | Enforce an average rate on the client |
| Network Term | Traffic Shaping | Traffic Policing |
| Request Fate (over limit) | Queued (if space) or dropped | Rejected immediately |
The Fundamental Trade-off
The Leaky Bucket prioritizes downstream stability at the cost of upstream latency. Requests don't fail — they wait. But that waiting introduces latency, and if the queue fills up, requests are dropped with no retry.
The Token Bucket prioritizes client experience at the cost of downstream predictability. Bursts are allowed, which means your downstream service must be able to handle the maximum burst size (capacity tokens) arriving simultaneously. If it can't, you've just moved the problem downstream.
Which one do real systems use?
Most public APIs — AWS, Stripe, GitHub, Twilio — use Token Bucket or a close variant. The reason is user experience: human and application traffic is inherently bursty, and rejecting valid requests because they arrived slightly clustered creates unnecessary friction.
Network hardware (routers, switches, traffic shapers) often uses Leaky Bucket because the downstream medium (a network link) truly cannot absorb bursts — every packet must be transmitted at the wire rate.
Many production systems use both in combination: a Token Bucket at the API gateway to allow short bursts, and a Leaky Bucket internally to smooth the load before it hits a sensitive database or queue.
If an interviewer asks "which algorithm does Nginx/HAProxy/AWS API Gateway use?", the honest answer is: a variant of Token Bucket, often called a "leaky bucket as a meter" (confusingly). AWS API Gateway's throttling is Token Bucket-based: it has a burst limit (bucket capacity) and a steady-state rate (refill rate). Knowing this distinction — and that the naming in documentation is sometimes inconsistent — signals real-world experience.