Cluster Autoscaler (CAS): The "ASG Manager

How CAS Works

The Cluster Autoscaler (CAS) does not launch EC2 instances directly. It acts as a middleman between Kubernetes and AWS Auto Scaling Groups (ASGs) — a native AWS construct that manages a fleet of identically configured EC2 instances.

The CAS loop works like this:

1. Watch: CAS detects one or more Pending pods.
2. Simulate: It iterates through your configured ASGs and asks: "If I added one node from ASG 'NodeGroup-A', would that pod become schedulable?" It runs this simulation using the ASG's declared instance type and labels.
3. Command: If the answer is yes, it calls the AWS Auto Scaling API to increment DesiredCapacity by 1.
4. Wait: AWS launches the instance (~1–2 min), the OS boots, the kubelet starts, the node registers with the cluster, and finally the scheduler places the pod.

Scale-down follows a similar pattern: CAS periodically checks whether any node's pods could fit on other existing nodes. If so, it cordons the node (marks it unschedulable), drains the pods, and decrements the ASG's DesiredCapacity.

The Limitations of CAS

Rigid instance selection. Each ASG is pre-configured with a single instance type (or a limited set via Mixed Instance Policies). If your ASG uses m5.large, CAS must launch an m5.large — even if your pod only needs 0.25 vCPU. The mismatch between pod requirements and available instance types is the root cause of most compute waste in CAS-managed clusters.

Operational complexity. To get meaningful flexibility, teams end up managing many Node Groups: one for general purpose, one for compute-optimized, one for memory-optimized, one for GPU, one for Spot, one for ARM. Each group requires its own ASG, launch template, IAM role, and CAS configuration. A mature cluster can have 15–30 Node Groups, each a potential source of misconfiguration.

Slow scale-up. The ASG indirection adds latency. The full path from Pending pod to Running pod typically takes 3–5 minutes. For latency-sensitive or bursty workloads, this is unacceptable.

Scale-down conservatism. CAS is intentionally cautious about removing nodes. It waits for a configurable idle period (default: 10 minutes) before terminating a node, and it will not remove a node if any pod on it lacks a controller (e.g., a standalone pod). This conservatism is safe but expensive.