Designing Systems That Don't Need to Change to Survive
The opposite of Control Plane dependency is Static Stability: the property of a system that allows it to continue operating correctly in its current state even when it cannot make any configuration changes.
A statically stable system does not need to call any API, launch any new resource, or update any configuration to handle a failure. It absorbs the failure using capacity and routing decisions that were already in place before the failure occurred.
The Fix for the Video Processing Scenario
Instead of reactively launching instances when CPU spikes, you provision enough capacity upfront to handle the loss of an entire Availability Zone — the largest single failure domain you should plan for.
Before the failure:
- AZ-A: 10 instances running at ~45% CPU
- AZ-B: 10 instances running at ~45% CPU
- Total capacity: 200% of normal load
During the failure (AZ-A goes down):
- The Load Balancer (Data Plane) detects failed health checks and stops routing to AZ-A. No API call required.
- AZ-B: 10 instances now running at ~90% CPU
- Service is degraded but alive. No RunInstances call was made.
Result: No API calls. No Control Plane interaction. The service survived because the capacity was already there.
The Trade-off
Static stability costs money. Running at 50% utilization to absorb an AZ failure means you are paying for twice the compute you use under normal conditions. This is a deliberate, explicit trade-off: you are buying reliability with money. The alternative — relying on the Control Plane to bail you out during a crisis — is cheaper in normal times but catastrophically expensive when it fails.
The engineering decision is: what is the cost of downtime vs. the cost of over-provisioning? For a payment processor, the answer is obvious. For a development environment, it is not.
When asked "How do you design for high availability?", most candidates answer with "multi-AZ" or "auto-scaling." The senior-level answer adds: "I design for static stability — I pre-provision enough capacity so that the system can survive an AZ failure without needing to call any AWS API. Auto-scaling is for efficiency, not for reliability."