CPU Profiling (On-CPU)
CPU profiling answers: "What is my code doing right now?"
- Method: Statistical sampling. At 99 Hz, the profiler checks which function is currently executing on each CPU core. After enough samples, the distribution of samples across functions approximates the distribution of CPU time.
- Why 99 Hz and not 100 Hz? Using a prime number (99) avoids accidentally synchronizing with application timers that fire at round numbers (100 Hz, 1000 Hz), which would cause sampling bias.
- Use cases: Reducing latency, identifying hot loops, reducing cloud compute costs by finding code that burns CPU unnecessarily.
Memory Profiling: Allocations vs. In-Use
Memory profiling answers two different questions, and it is important to know which one you are asking:
Allocation profiling ("Where is memory being created?"):
* Tracks every call to malloc, new, or equivalent allocators.
* Shows allocation churn: code that creates and immediately discards large numbers of short-lived objects. Even if memory usage looks stable (objects are freed quickly), high allocation rates hurt performance because the garbage collector must constantly run to reclaim them.
* Use case: Reducing GC pressure, reducing CPU time spent in the allocator.
Heap / In-Use profiling ("Where is memory being held?"):
* Tracks memory that was allocated but has not yet been freed.
* This is how you find memory leaks: a function that allocates objects and never releases them, causing the heap to grow until the process is OOM-killed.
* Use case: Diagnosing OOMKilled pods, finding unbounded caches, finding leaked goroutines holding references.
eBPF and memory profiling:
CPU profiling with eBPF is mature and low-overhead. Memory profiling with eBPF is more complex — hooking malloc requires uprobes (userspace probes) rather than perf events, and the overhead is higher because allocations happen far more frequently than timer ticks. Most production deployments use eBPF for CPU profiling and language-specific agents for detailed memory profiling.
Interviewers often ask: "Your Go service's memory usage grows by 50 MB per hour and never shrinks. How do you diagnose it?" The answer is heap profiling, not allocation profiling. Allocation profiling shows churn (fast create/destroy). Heap profiling shows what is still alive — the leak. In Go, you expose net/http/pprof and hit /debug/pprof/heap to get a heap profile, then load it in go tool pprof to see which functions hold live memory.