Introduction: Why "grep" Isn't Enough Anymore

The Narrative of Your System

Think of your application as a hospital. Metrics are the vital signs monitors on the wall — they tell you a patient's heart rate is dangerously low. But logs are the nurse's notes: "Patient complained of chest pain at 9:47 AM, administered medication at 9:52 AM, heart rate stabilized at 10:01 AM." One tells you that something is wrong; the other tells you what happened and why.

A log is a timestamped record of a discrete event that occurred within your application. "User logged in." "Database connection failed after 3 retries." "Payment of $150 processed successfully for order #8821." Each log line is a sentence in the ongoing story of your system's behavior.

The Scale Problem

In the old days of a single monolithic server, debugging was simple: SSH in and run grep "ERROR" /var/log/myapp.log. The log file was right there, on one machine, and it persisted indefinitely.

In a modern cloud-native environment, this approach collapses entirely. Consider what you're actually dealing with:

• Dozens of microservices, each writing their own logs in their own format.
• Hundreds of short-lived containers, each with its own ephemeral filesystem. When a container crashes and restarts, its local logs are gone.
• Multiple nodes spread across availability zones or even regions.
• Thousands of requests per second, each touching multiple services.

The container that logged the error you're investigating might not even exist anymore by the time your alert fires. You need a fundamentally different approach: collect logs from every source the moment they're written, ship them to a central system, and make them searchable at scale.