Free — no signup required

Deep Dive: Space Amplification

2 min read

B-Tree Space Amplification (Fragmentation)

B-Tree pages are rarely 100% full. When a page splits, two half-full pages replace one full page — instantly dropping utilization to ~50%. As more data is inserted, pages fill back up, but the average utilization across a real-world B-Tree is typically 60–70%.

This means for every 100 GB of actual data, a B-Tree database uses approximately 130–150 GB of disk space — a Space Amplification of 1.3–1.5x. This is relatively modest and predictable.

PostgreSQL adds another layer: its MVCC (Multi-Version Concurrency Control) implementation stores old row versions (called "dead tuples") in the heap until VACUUM cleans them up. Under heavy update workloads, dead tuples can cause table bloat, temporarily pushing Space Amplification to 2x or higher until VACUUM runs.

LSM Tree Space Amplification (Ghost Data)

LSM Trees are append-only. When you delete a record, the engine does not remove the old data — it writes a special marker called a Tombstone: {user_id: 123, deleted: true, timestamp: T3}. The original record still occupies space on disk in an older SSTable.

Until compaction runs and merges the levels containing both the tombstone and the original record, both exist on disk simultaneously. For a workload with heavy deletes or updates:

Disk State Before Compaction:
  L2 SSTable: {user_id: 123, email: "[email protected]"}  ← original, 50 bytes
  L0 SSTable: {user_id: 123, deleted: true}              ← tombstone, 20 bytes

  Logical data size: 0 bytes (record is deleted)
  Physical disk usage: 70 bytes
  Space Amplification: ∞ (storing data that doesn't logically exist)

In practice, Space Amplification for LSM Trees ranges from 1.1x (after aggressive compaction on an insert-only workload) to 3x or more (on a heavy update/delete workload with infrequent compaction). Time-series databases that use LSM Trees often mitigate this by using time-windowed compaction — SSTables are organized by time window, and entire windows are dropped atomically when data expires, avoiding tombstone accumulation entirely.

This is one of 18 chapters

Every chapter is free — Kubernetes, Terraform, SRE, distributed systems, and more — with fast daily review built in.

Sign up free