Async/Await and Tokio: Concurrency for Ops

The Waiting Problem

Infrastructure tools spend most of their time waiting. Waiting for an HTTP response from the Kubernetes API server. Waiting for a DNS lookup. Waiting for a file to flush to disk. Waiting for a TCP connection to establish. If your program blocks a thread during each wait, you need one thread per concurrent operation — and threads are expensive (typically 1–8 MB of stack memory each).

The solution is asynchronous I/O: instead of blocking a thread while waiting, the program registers interest in an event ("tell me when this socket has data") and moves on to other work. When the event fires, execution resumes. This allows a single thread to manage thousands of concurrent operations.

Rust provides the async/await syntax for writing asynchronous code that looks sequential but executes concurrently:

async fn fetch_pod_count(client: &Client) -> u32 {
    let pods = client.list_pods().await; // Suspends here, doesn't block the thread
    pods.len() as u32
}

Tokio: The Missing Runtime

Unlike JavaScript (which has a built-in event loop) or Go (which has a built-in scheduler for goroutines), Rust's standard library deliberately does not include an async runtime. It provides the async/await syntax, but you need a community crate to actually drive the execution — to poll futures, manage the event loop, and schedule work across threads.

Tokio is the de-facto standard async runtime for Rust. It provides:
- A multi-threaded work-stealing scheduler (similar to Go's goroutine scheduler)
- Async versions of I/O primitives: TCP, UDP, files, timers
- Utilities for spawning tasks, channels, and synchronization

A minimal Tokio-based HTTP server skeleton looks like this:

use tokio::net::TcpListener;

#[tokio::main] // This macro sets up the Tokio runtime
async fn main() {
    let listener = TcpListener::bind("0.0.0.0:8080").await.unwrap();
    loop {
        let (socket, _) = listener.accept().await.unwrap();
        tokio::spawn(async move {
            // Handle each connection in its own async task
            handle_connection(socket).await;
        });
    }
}

Linkerd2-proxy uses Tokio to handle tens of thousands of concurrent connections per proxy instance, with memory usage measured in megabytes rather than gigabytes. This is the practical payoff of Rust's async model for infrastructure.