Seeing Inside the Black Box
One of the most immediate practical benefits of deploying a service mesh is the observability it provides — for free, without any changes to application code.
Because every request passes through an Envoy sidecar, the mesh can collect data across all three observability pillars:
Metrics
Envoy automatically emits metrics for every request it proxies: request count, error rate, latency percentiles (p50, p95, p99), and bytes transferred. These metrics are labeled with source and destination service names, so you can instantly answer questions like "What is the p99 latency from frontend to payment-service?" without any instrumentation in your application code. These metrics are typically scraped by Prometheus and visualized in Grafana dashboards.
Distributed Tracing
When a request enters the mesh, Envoy can generate a trace span — a record of that request's journey through the system. As the request hops from service to service, each Envoy sidecar adds its own span, creating a complete trace that shows exactly where time was spent. This integrates with tracing backends like Jaeger or Zipkin. One important caveat: for traces to be stitched together correctly across services, applications must forward trace headers (like x-b3-traceid) from incoming requests to outgoing ones. This is the one piece of observability that requires minimal application cooperation.
Access Logs
Every request proxied by Envoy can be logged with full detail: source IP, destination service, HTTP method, URL, response code, and latency. These logs can be streamed to any log aggregation system (Elasticsearch, Loki, Splunk) without any logging library in the application.
The combination of these three signals — metrics for alerting, traces for debugging, logs for forensics — gives platform teams a complete picture of service behavior that would otherwise require significant per-service instrumentation effort.
Interviewers sometimes ask: "What is the one observability feature in a service mesh that still requires application code changes?" The answer is distributed tracing header propagation. Envoy can generate and inject trace headers on the first hop, but for the trace to be correctly correlated across multiple service hops, each application must read the incoming trace headers (x-b3-traceid, x-b3-spanid, etc.) and forward them on outbound requests. The mesh cannot do this automatically because it does not understand the application's internal call graph. Everything else — metrics, access logs, and even the initial trace span — is fully automatic.