The Problem: Abrupt Death (and Abrupt Birth)
By default, when an ASG scales in, it terminates the instance immediately. Active TCP connections are severed. Local data is lost. Any in-flight work is abandoned. Conversely, when an ASG scales out, the new instance is added to the load balancer the moment it passes a health check — even if the application process is still warming up its caches and not yet ready to serve real traffic.
Lifecycle Hooks solve both problems by inserting a controlled pause into the ASG state machine.
The Two Hook Types
autoscaling:EC2_INSTANCE_LAUNCHING (Pending:Wait)
Fires before the instance is added to the load balancer. Use this to:
- Download large model files or datasets from S3
- Register the instance with a service discovery system (e.g., Consul)
- Run integration tests to verify the instance is healthy before it receives traffic
autoscaling:EC2_INSTANCE_TERMINATING (Terminating:Wait)
Fires before the instance is destroyed. Use this to:
- Upload logs or state to S3 or DynamoDB
- Drain in-flight database connections gracefully
- Deregister from service discovery
- Upload a game save file before a game server instance is recycled
The Complete State Machine
How the Notification Flow Works
- Event: ASG decides to terminate Instance X.
- Pause: Instance X enters
Terminating:Wait. It stays here for up to 1 hour by default (configurable from 30 seconds to 7200 seconds). - Notification: ASG publishes an event to EventBridge (recommended), SNS, or SQS. The event payload includes the instance ID, the hook name, and a lifecycle action token — a unique identifier for this specific pause event.
- Logic: A Lambda function receives the event. It uses SSM Run Command to execute a script on the instance (no SSH required). The script does its work — uploads logs, flushes state, etc.
- Proceed: The Lambda calls
complete-lifecycle-actionwith the lifecycle action token, telling the ASG: "I'm done, you may proceed." The instance is then terminated.
Registering the hook means attaching it to the ASG for the terminating transition, pointing it at a notification target (EventBridge, SNS, or SQS) with an IAM role that has permission to publish there, and setting a heartbeat timeout plus a default result. Later, once the Lambda has finished its cleanup work, it calls back with the hook name, ASG name, and the lifecycle action token it received in the event, telling the ASG to CONTINUE — which releases the pause and lets termination proceed.
The default-result Parameter
This is a critical detail. If the hook timeout expires before complete-lifecycle-action is called (e.g., your Lambda crashes), the ASG falls back to default-result:
- CONTINUE — proceed with the launch or termination as normal.
- ABANDON — for a scale-out hook, terminate the instance; for a scale-in hook, terminate immediately without waiting further.
For a termination hook, CONTINUE is usually the right default — you'd rather terminate the instance than leave it running forever if your cleanup Lambda fails.
A common interview scenario: "Your ASG is scaling in but instances are staying in Terminating:Wait for an hour before dying. What's wrong?" The answer is that complete-lifecycle-action is never being called — either the Lambda is failing silently, the EventBridge rule is misconfigured, or the IAM role lacks autoscaling:CompleteLifecycleAction permission. The fix is to add CloudWatch Logs to the Lambda and verify the EventBridge rule target is correctly configured.