The Interruption Signal
When AWS needs a Spot instance back, it sends an EC2 Instance Interruption Notice — a metadata event that appears two minutes before the instance is forcibly terminated. Two minutes is not much time, but it's enough to drain a node gracefully if your tooling is watching for it.
Legacy setups often missed this signal entirely. The instance would simply stop responding, Kubernetes would wait for its node health check timeout (default: 5 minutes), and only then would it reschedule the pods — meaning 7+ minutes of potential downtime for anything running on that node.
Karpenter's Interruption Handler
Karpenter integrates with an Amazon SQS queue that receives interruption events from AWS EventBridge in real time. The moment AWS decides to reclaim an instance, the event lands in the queue — often 30–60 seconds before the 2-minute warning even starts.
The response sequence is:
- Cordon: Karpenter immediately marks the affected node as
Unschedulable. No new pods are placed there. - Surge: Karpenter launches a replacement node before the old one dies. It doesn't wait.
- Drain: Karpenter evicts pods from the condemned node. Because the replacement is already booting (or already ready), pods have somewhere to land immediately.
- Terminate: The old node shuts down. By this point, all workloads have already migrated.
What You Need to Set Up
The SQS-based interruption handling requires infrastructure outside the cluster:
EventBridge Rule → SQS Queue → Karpenter Controller (reads queue)
In practice, this is typically deployed via the Karpenter Helm chart with the --set controller.settings.aws.interruptionQueueName=<queue-name> flag, or configured in the Karpenter controller's ConfigMap. The SQS queue and EventBridge rules are usually created by the same Terraform or CloudFormation module that installs Karpenter.
A common interview question is: "How does Karpenter achieve near-zero downtime during Spot interruptions?" The answer has two parts: (1) it uses an SQS queue fed by EventBridge to receive interruption notices before the 2-minute warning expires, and (2) it launches a replacement node concurrently with draining the old one (surge-before-drain), so pods have somewhere to go immediately. Contrast this with naive approaches that wait for the node to become NotReady before acting — by then, you've already lost 5+ minutes.
Pod Disruption Budgets: Your Safety Net
Karpenter respects Pod Disruption Budgets (PDBs). A PDB tells Kubernetes: "Never have fewer than N healthy replicas of this deployment at any time."
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: my-service-pdb
spec:
minAvailable: 2
selector:
matchLabels:
app: my-service
During a drain, Karpenter will not evict a pod if doing so would violate the PDB. This means if your replacement node isn't ready yet and evicting a pod would drop you below minAvailable: 2, Karpenter waits. PDBs are the last line of defense ensuring graceful handling even when the surge node is slow to boot.