"Recover Automatically from Infrastructure or Service Disruptions"
Reliability is the pillar most directly tied to uptime. The core insight is a shift in mindset: stop asking "how do I prevent failures?" and start asking "how do I recover from failures automatically?" In a distributed system at scale, individual component failures are not exceptional events — they are routine. A reliable system is one that absorbs those failures without the user noticing.
Think of it like a highway with multiple lanes. If one lane is blocked, traffic flows around it. A single-lane road with a blockage stops everything.
Design Principles
- Automatically recover from failure: Monitor your system for key health indicators. If a server fails its health check, terminate it and replace it automatically using Auto Scaling. The system heals itself without a human waking up at 3am.
- Test recovery procedures: Don't wait for a real disaster to discover your backup restoration takes 6 hours and fails halfway through. Run Chaos Engineering experiments — deliberately inject failures in a controlled way to verify your recovery automation actually works.
- Scale horizontally to increase aggregate availability: Replace one large server with many small ones. If one large server fails, you lose 100% capacity. If one of ten small servers fails, you lose 10% capacity and the other nine absorb the load.
- Stop guessing capacity: In the cloud, you don't need to provision for peak load 24/7. Use Auto Scaling to match capacity to actual demand automatically.
- Manage change through automation: Changes to infrastructure should go through a pipeline with testing, not be applied manually. Manual changes are the leading cause of outages.
The Difference Between Availability and Durability
These terms are often confused. Availability means the system is accessible and responding right now (e.g., "99.99% uptime"). Durability means data is not lost (e.g., S3 offers 99.999999999% — "11 nines" — durability, meaning data is extremely unlikely to be lost). A system can be highly available but have low durability (data is accessible but gets corrupted), or highly durable but have low availability (data is safe but the service is down for maintenance). Well-architected systems require both.
Key Services: Auto Scaling, Elastic Load Balancing, Multi-AZ RDS, S3 Cross-Region Replication, Route 53 (health checks and DNS failover), AWS Backup.