The Speed Gap
In computing, there is a massive difference in speed between different types of storage. Think of it like the difference between remembering your own phone number (instant) versus looking it up in a phone book (slow). Your brain caches frequently needed information so you don't have to look it up every time.
- Reading from memory (RAM) takes nanoseconds (billionths of a second).
- Reading from a disk (SSD/HDD) or across a network to a database takes milliseconds (thousandths of a second).
A millisecond is 1,000,000 times longer than a nanosecond. If fetching data from memory took 1 second, fetching it from disk would take almost 12 days. That gap is not a rounding error — it is the entire motivation for caching.
What is a Cache?
A cache is a high-speed data storage layer that holds a subset of data — typically the most recently or frequently accessed subset — so that future requests for that data are served faster than fetching it from the primary storage location. The cache trades storage capacity (it holds less data) for speed (it returns data far faster).
As a Database Reliability Engineer (DBRE), you often deal with databases under heavy read load. A single popular query — say, "fetch the top 10 trending products" — might be executed thousands of times per second by different users. Without caching, each of those requests hits the database, burns CPU, and competes for I/O. With caching, the result of that query is computed once, stored in fast memory (using a tool like Redis or Memcached), and served directly to the next 9,999 requests without touching the database at all.
The Cache Hit Rate
The effectiveness of a cache is measured by its hit rate: the percentage of requests that are served from the cache rather than the database. A hit rate of 95% means only 5% of requests reach the database — a 20x reduction in database load. A hit rate of 50% means half your requests still hammer the database, which may not be enough relief. Monitoring hit rate is the first thing you check when evaluating whether a cache is actually helping.