The Hardware Cache You've Never Heard Of
Every time your CPU accesses a memory address, it must translate that virtual address (what the application sees) into a physical address (the actual location in RAM chips). This translation requires consulting the page table — but the page table lives in RAM, which means every memory access would require an additional RAM lookup just to find where the real data is. That would halve your effective memory bandwidth.
To avoid this, CPUs include a small, extremely fast hardware cache called the Translation Lookaside Buffer (TLB). The TLB stores recently used virtual-to-physical address mappings directly inside the CPU. A TLB hit means the translation is instant — no RAM access needed. A TLB miss means the CPU must "walk" the page table in RAM, which takes 10–100 nanoseconds and stalls the pipeline.
The critical constraint: the TLB has a fixed number of entries, typically 64–1024 depending on the CPU model. With 4KB pages, a TLB with 512 entries can cover at most 512 × 4KB = 2MB of memory at once. A database with a 64GB working set will experience near-constant TLB misses, because its working set is 32,000 times larger than what the TLB can cover.
With 2MB HugePages, those same 512 TLB entries cover 512 × 2MB = 1GB of memory. The TLB's effective reach increases by 512x. For a database whose hot working set fits within 1GB, TLB misses essentially disappear. This is the core mechanism behind HugePage performance gains — not magic, just better hardware utilization.
TLB Coverage Comparison (512-entry TLB):
4KB pages: 512 × 4KB = 2MB coverage
2MB pages: 512 × 2MB = 1,024MB coverage (512x improvement)
1GB pages: 512 × 1GB = 524,288MB coverage (262,144x improvement)
Interviewers often ask: "Why do HugePages improve database performance?" A weak answer is "fewer page table entries." The strong answer explains the TLB: "HugePages increase the TLB's effective memory coverage by 512x. For a database with a large working set, this eliminates TLB misses — each of which costs 10–100ns and stalls the CPU pipeline. At millions of memory accesses per second, eliminating TLB misses translates directly to throughput gains of 5–15%."