The Problem: Hot Partitions
Imagine a physical warehouse where all packages addressed to "New York" go to a single loading dock. On Black Friday, that dock gets overwhelmed while every other dock sits idle. This is exactly what happens with a hot partition in DynamoDB.
DynamoDB distributes data across internal storage partitions using consistent hashing on the Partition Key. Each partition supports a maximum of 1,000 Write Capacity Units (WCUs) per second and 3,000 Read Capacity Units (RCUs) per second. If a single Partition Key value receives more traffic than one partition can handle, DynamoDB throttles those requests — meaning writes are rejected with a ProvisionedThroughputExceededException error.
For a voting app during a live TV show, if 1 million people vote for "Singer A" in 60 seconds, that's roughly 16,667 writes/second — all targeting PK=CANDIDATE#A. One partition cannot handle this. Votes are dropped.
The Solution: Write Sharding
Write sharding artificially distributes a logically single item across multiple physical Partition Keys, called shards. Each shard is an independent item in DynamoDB and receives a fraction of the total write traffic.
How it works:
- Determine N (number of shards): Divide your peak write rate by the per-partition limit, then apply a safety multiplier.
- Write path: When recording a vote, the application generates a random integer between
1andNand appends it to the Partition Key. - Read path: To get the total, query all N shards and aggregate in application code.
Write path example:
ShardID = random_int(1, N)
PK = "CANDIDATE#A#" + str(ShardID)
# Writes are distributed across CANDIDATE#A#1 through CANDIDATE#A#N
Read path example:
total_votes = 0
for shard_id in range(1, N+1):
item = GetItem(PK="CANDIDATE#A#" + str(shard_id))
total_votes += item["VoteCount"]
The Trade-offs
| Concern | Impact |
|---|---|
| Write throughput | Scales linearly with N — 10 shards = 10,000 WCUs/sec capacity. |
| Read complexity | Reads require N GetItem calls (or a Query with prefix filter) plus client-side aggregation. |
| Read latency | Can parallelize all N reads; total latency ≈ latency of one read, not N × latency. |
| Consistency | Aggregated totals are eventually consistent across shards if using parallel async reads. |
| Cost | N shards = N items = N × storage cost. Negligible for small N. |
Senior insight: Write sharding is a write-time decision with permanent read-time consequences. Choose N conservatively — you cannot easily re-shard later without a migration. A common pattern is to use time-bounded sharding: reset shard counters each hour or day, then aggregate historical totals into a single "archive" item. This bounds the read fan-out over time.
A common interview question is: "How would you handle a hot partition in DynamoDB?" Weak answers mention "increase provisioned capacity." Strong answers explain write sharding: distribute writes across N logical shards using a random suffix on the PK, then aggregate across shards on read. Follow up by discussing how to choose N and the read-time aggregation cost.