When You Can't Even Get a Shell
If the instance is running but you cannot SSH in at all, you're in a black-box scenario. You need to diagnose without a terminal session.
Systematic checklist:
1. Security Group / Firewall Rules
The most common cause by far. Verify that port 22 is open to your IP address in the instance's security group (AWS) or firewall rules (GCP/Azure). A misconfigured security group looks identical to a broken instance from the outside.
2. SSH Key Injection Failure
If cloud-init failed to fetch the SSH public key from the metadata service (e.g., network wasn't ready, or you specified the wrong key pair name), it never wrote anything to ~/.ssh/authorized_keys. The instance is running fine — it just won't accept your key. You'll see Permission denied (publickey) in your SSH client.
3. The Serial Console / System Log
Every major cloud provider exposes the instance's serial console output — the text that would appear on a physical monitor during boot. This is your most powerful black-box diagnostic tool.
- AWS: EC2 Console → select instance → Actions → Monitor and troubleshoot → Get system log
- GCP: Compute Engine → select instance → Serial port 1 (console)
- Azure: Virtual Machine → Boot diagnostics → Serial log
What to look for in the serial log: a line confirming cloud-init "finished," naming its datasource and elapsed time, means it completed. A Python traceback means it crashed. A kernel panic or "no bootable device" message means the problem is at the OS level — cloud-init never even got the chance to run.
4. Instance Never Terminates on Failure
A critical operational detail: if your User Data script exits with a non-zero code, or even if cloud-init itself errors out, the instance does not terminate. It keeps running in a broken state, accruing cost. You must manually terminate it, fix your User Data, and re-launch. This is why validating your cloud-config against cloud-init's schema before you ever launch the instance is so important.