Karpenter: Just-in-Time Provisioning

The Paradigm Shift

Karpenter eliminates the ASG middleman entirely. Instead of managing pre-configured node groups, it reads pod requirements directly and queries the full AWS EC2 instance catalog to find the optimal instance type at that exact moment. It then calls the EC2 RunInstances API directly — no ASG involved.

This is called Just-in-Time (JIT) provisioning: capacity is selected and created precisely when needed, sized precisely for what is needed.

How Karpenter Works

1. Watch: Karpenter's controller detects Pending pods via the Kubernetes API.
2. Analyze: It reads each pod's resource requests (cpu, memory, nvidia.com/gpu), node selectors, affinities, tolerations, and topology spread constraints.
3. Select: It queries the EC2 instance catalog — hundreds of instance types — and finds the cheapest option that satisfies all pod requirements. It considers Spot vs. On-Demand pricing in real time.
4. Launch: It calls ec2:RunInstances directly, passing a pre-built launch template. The instance boots, the node joins the cluster, and pods are scheduled.

Karpenter's Configuration Model

Karpenter uses two Custom Resource Definitions (CRDs):

NodePool — defines what Karpenter is allowed to provision. Think of it as the policy document.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: karpenter.k8s.aws/instance-family
          operator: In
          values: ["m", "c", "r"]
        - key: karpenter.k8s.aws/instance-size
          operator: NotIn
          values: ["nano", "micro", "small"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
  limits:
    cpu: 1000
    memory: 1000Gi
  disruption:
    consolidationPolicy: WhenUnderutilized
    consolidateAfter: 30s

EC2NodeClass — defines how instances are configured: AMI, subnet, security groups, instance profile, user data.

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiSelectorTerms:
    - alias: al2023@latest
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: my-cluster
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: my-cluster
  role: KarpenterNodeRole-my-cluster

Why It Wins

Speed. Removing the ASG layer cuts scale-up time from 3–5 minutes to 60–90 seconds in most cases. Karpenter also batches pending pods — it waits a configurable batchingWindow (default: 1 second) to collect all pending pods before making a single, optimized provisioning decision.

Flexibility without complexity. A single NodePool can authorize hundreds of instance types. Karpenter selects the right one per workload. You no longer need 20 Node Groups to get 20 different instance shapes.

Cost awareness. Karpenter natively understands Spot pricing and interruption rates. It can prefer Spot instances and automatically fall back to On-Demand if Spot capacity is unavailable — all within a single NodePool.