Observability & SRE

Introduction: What Are Metrics?

The Four Core Metric Types

Metric Types: Measuring Distributions (Histograms & Summaries)

The Power of Labels

The Danger of High Cardinality

Collecting Metrics: Push vs. Pull

Exporters: The Universal Adapters

Introduction: Why "grep" Isn't Enough Anymore

Structured vs. Unstructured Logging

The Aggregation Challenge: ELK vs. Loki

Log Shipping: Getting Logs from Containers to the Backend

Context Injection: Connecting the Dots

Log Levels: Signal vs. Noise

Log Sampling: Handling the Flood

Introduction: The Microservice Detective

The Building Blocks: Spans and Traces

Spans vs. Events

OpenTelemetry (OTel): The Industry Standard

Context Propagation: Connecting the Dots

Sampling Strategies: Handling the Flood

Introduction: How Reliable is "Reliable Enough"?

SLI (Service Level Indicator): The Measurement

SLO (Service Level Objective): The Goal

SLA (Service Level Agreement): The Contract

The Error Budget: The Currency of Risk

Burn Rates and Multi-Window Alerting

Introduction: The "Check Engine" Light

Symptom-Based vs. Cause-Based Alerting

Managing the Flood: Grouping and Deduplication

Routing: Getting the Alert to the Right Person

Fatigue Management: Silences and Inhibition

Alert Rules in Practice: Anatomy of a Prometheus Alert

Introduction: Why We Inject Failure

The Scientific Method of Chaos

Step 1: The Steady State Hypothesis

Step 2: Fault Injection (The Attacks)

GameDays: Learning Together

Tools of the Trade

Introduction: It doesn't matter if the server is fast...

Two Ways to Measure: RUM vs. Synthetics

Core Web Vitals: The Google Standard

LCP (Largest Contentful Paint)

CLS (Cumulative Layout Shift)

INP (Interaction to Next Paint)

The Debugging Gap: Source Maps

Containing the Blast: Error Boundaries

The Frontend Observability Stack

Introduction: The "It Works on My Machine" Problem

How It Works: Sampling and Stack Traces

The Visualization: Flamegraphs

The Tools: Pyroscope and Parca

Memory Profiling: It's Not Just CPU

Connecting Profiling to the Rest of Your Observability Stack

Introduction: The "Direct to Vendor" Trap

The Architecture: Agents and Aggregators

Strategy 1: Cost Control via Filtering and Sampling

Strategy 2: Intelligent Routing

Tool Spotlight: Vector vs. Fluent Bit

The OpenTelemetry Collector: The Vendor-Agnostic Standard

Introduction: The Economics of Tracing

Trace Coherency: The Golden Rule

Head-Based Sampling: The Coin Flip

Tail-Based Sampling: The Time Machine

The Architecture Challenge: Scaling Tail Sampling

Buffering Challenges: RAM vs. Latency

Hybrid Sampling Strategies: Combining Head and Tail

Introduction: The "All Systems Green" Fallacy

Mapping Technical Metrics to Business KPIs

Revenue-at-Risk: Quantifying Downtime

Anomaly Detection: Handling Seasonality

The Business Observability Dashboard

Introduction: The "Needle in the Haystack" Problem

What is an Exemplar?

The OpenMetrics Standard

The User Experience in Grafana

The Instrumentation Layer: Connecting Traces to Metrics

Exemplars in the Broader Observability Picture

Introduction: It's Not Just a Job Title

The Anti-Pattern: The "Kitchen Sink" SRE

Model 1: Embedded SRE (The Co-Pilot)

Model 2: Platform SRE (The Utility Provider)

Model 3: Consulting SRE (The Mercenaries)

Hybrid Models: Combining Approaches

Introduction: The Language of Time Series

Basic Selectors: Finding the Needle

Rate vs. Increase: The Most Common Confusion

Aggregation: Summing Up

Histograms and Quantiles: The Hard Math

Arithmetic and Multi-Metric Queries

Grafana Variables: Build Once, Use Everywhere

Grafana Annotations: Correlating Events with Metrics

Introduction: "It's Always the Network

Strategy 1: Active Monitoring (Synthetics)

Strategy 2: Passive Monitoring (Packet Capture)

The Invisible Enemy: Micro-bursts

Detecting Micro-bursts

Key NPM Metrics Reference

Introduction: The Tower of Babel Problem

What Are Semantic Conventions?

Resource Attributes: Defining the "Who

Unified Service Tagging: The Golden Trio

Span and Metric Attributes: Defining the "What

The Semantic Conventions Package: Using Constants in Code

Conventions Evolve: Handling Breaking Changes

Introduction: The Black Box Problem

Continuous Profiling: The "Security Camera

How eBPF Actually Works: The Kernel Hook

Zero Instrumentation: No Code Changes Required

The Visualization: Reading Flamegraphs

Parca: The eBPF Specialist

Pyroscope: The Polyglot Profiler

CPU vs. Memory Profiling

Introduction: The Multiplication Problem

Detecting the Culprit

Mitigation Strategy 1 — Remove the Label

Mitigation Strategy 2 — Aggregate Before Storage

Introduction: The "Grep" Dead End

The Power of Querying

Implementing Semantic Logging

Schema Discipline: The "Semantic" Part

Context Propagation and Log Correlation