The Fragmentation Problem
Before OpenTelemetry, every observability vendor (Datadog, New Relic, Zipkin, Jaeger) had its own proprietary SDK and wire format. If you instrumented your application with Zipkin's library and later wanted to switch to Datadog, you had to rewrite every instrumentation call in your codebase. Vendor lock-in was baked into the instrumentation layer itself.
OpenTelemetry (OTel) is an open-source project under the Cloud Native Computing Foundation (CNCF) that solves this by standardizing how telemetry data — metrics, logs, and traces — is generated, collected, and exported. It is not a storage backend or a visualization UI. It is the plumbing: the pipes, valves, and connectors that get data from your application to whatever backend you choose.
Instrument once with OTel. Send to any backend. Switch backends without touching application code.
OTel Architecture Components
1. SDKs (Software Development Kits)
Libraries you add to your application code, available for every major language (Java, Python, Go, Node.js, .NET, Ruby, etc.). The SDK handles span creation, context propagation, and batching data for export.
OTel offers two instrumentation modes:
- Automatic instrumentation: Zero-code agents that hook into common frameworks (Spring Boot, Flask, Django, Express, gRPC) and generate spans automatically. You attach a Java agent JAR or a Python sitecustomize hook, and spans appear without writing a single line of tracing code.
- Manual instrumentation: You explicitly create spans in your code for custom business logic that auto-instrumentation doesn't cover.
2. The OTel Collector
A standalone binary (typically deployed as a sidecar or DaemonSet in Kubernetes) that acts as a vendor-agnostic telemetry pipeline. It has three stages:
- Receivers: Accept data from SDKs in various formats (OTLP, Zipkin, Jaeger).
- Processors: Transform the data — filter noisy spans, add resource attributes, perform tail-based sampling, batch for efficiency.
- Exporters: Send data to one or more backends (Jaeger, Grafana Tempo, Datadog, Google Cloud Trace).
3. Exporters
Plugins that know the wire format of a specific backend. Because exporters live in the Collector (not your app), you can reconfigure where data goes without redeploying your services.
Interviewers frequently ask: "What is OpenTelemetry and how does it differ from Jaeger or Zipkin?" A common mistake is saying OTel replaces Jaeger. The correct answer: OTel is the instrumentation and collection standard — it defines how data is generated and transported. Jaeger and Grafana Tempo are storage and visualization backends — they store and query the trace data. OTel sends data to Jaeger; it does not replace it. The analogy: OTel is the postal system, Jaeger is the warehouse where packages are stored and retrieved.