Every public API faces the same threat: a single misbehaving client — whether a buggy retry loop or a deliberate denial-of-service attack — can exhaust your backend capacity and cause failures for everyone else. API Gateway prevents this with the Token Bucket Algorithm, a standard rate-limiting technique used across distributed systems.
The Metaphor
Picture a physical bucket that holds tokens. Each token represents permission to process one request.
- The Bucket capacity = your Burst Limit (maximum tokens the bucket can hold).
- The refill rate = your Rate Limit (tokens added per second, i.e., Requests Per Second / RPS).
- Each incoming request consumes one token. If the bucket is empty, the request is rejected with HTTP 429 Too Many Requests.
A Concrete Walkthrough
Configuration: Rate = 10 RPS, Burst = 20 requests
| Time | Event | Tokens Before | Tokens After |
|---|---|---|---|
| T=0s | Bucket starts full | — | 20 |
| T=0s | 20 requests arrive simultaneously | 20 | 0 |
| T=0s | 21st request arrives | 0 | 429 rejected |
| T=0.1s | Refill tick (+1 token) | 0 | 1 |
| T=0.1s | 1 new request arrives | 1 | 0 |
| T=1s | 10 refill ticks | 0 | 10 |
The burst limit allows your API to absorb short traffic spikes (e.g., a mobile app where all users open it simultaneously at 9am) without rejecting requests, while the rate limit protects your backend from sustained overload.
Throttling Hierarchy
AWS applies throttling at three levels, from broadest to most specific:
- Account-level default: 10,000 RPS burst, 5,000 RPS rate — shared across ALL APIs in the account/region.
- Stage-level: You can set per-stage limits on a specific API (e.g., your production stage gets 1,000 RPS, your dev stage gets 100 RPS).
- Method-level: You can set per-method limits (e.g.,
POST /paymentsgets 50 RPS,GET /productsgets 500 RPS).
The most specific limit wins. If a method-level limit is set, it overrides the stage limit for that method.
Usage Plans & API Keys
To monetize your API or enforce per-client limits, you create Usage Plans:
- Silver Plan: 10 RPS, 500 requests/day.
- Gold Plan: 100 RPS, 10,000 requests/day.
- Platinum Plan: 500 RPS, unlimited requests/day.
You distribute API Keys to clients, associate each key with a Usage Plan, and AWS enforces these limits per key. This is how SaaS companies build tiered API products on top of API Gateway.