The Provisioned Cluster Problem
With a standard (provisioned) MSK cluster, you choose broker count and instance type upfront. This means you must predict your peak throughput and provision for it — even if your actual load is bursty or unpredictable. An event-driven application that spikes to 500 MB/sec for 10 minutes per day but idles at 1 MB/sec the rest of the time still pays for a cluster sized for 500 MB/sec around the clock.
MSK Serverless
MSK Serverless is a cluster mode where you don't provision brokers at all. You create a cluster, configure topics, and MSK automatically scales the underlying infrastructure to match your throughput. You pay per GB of data written and read, not for idle broker capacity.
How it works internally: MSK Serverless abstracts the broker layer entirely. AWS manages a pool of shared infrastructure and allocates capacity to your cluster on demand. From your application's perspective, the bootstrap endpoint behaves identically to a provisioned cluster.
Limits and trade-offs:
| Dimension | MSK Serverless | Provisioned MSK |
|---|---|---|
| Capacity planning | None required | Must size upfront |
| Max throughput | 200 MB/sec ingress per cluster | Scales with broker count/type |
| Pricing model | Per GB in/out | Per broker-hour + EBS |
| Kafka version control | AWS-managed | You choose version |
| KRaft support | Yes | Yes (new clusters) |
| Best for | Bursty, unpredictable workloads | Steady, high-throughput workloads |
When to choose Serverless: Development environments, event-driven microservices with variable load, new applications where throughput is unknown. When throughput exceeds ~100 MB/sec consistently, provisioned clusters typically become more cost-effective.