Why Infrastructure Drifts
Imagine you carefully arrange your desk on Monday morning — everything in its exact right place. By Friday, pens have migrated, papers have piled up, and someone borrowed your stapler. Your desk has drifted from its intended state. Infrastructure does the same thing, except the consequences are security breaches, outages, and compliance failures rather than a missing stapler.
In infrastructure terms, drift is the divergence between your Desired State — what your code says should exist — and the Actual State — what is actually running in your cloud environment. Every IaC workflow assumes these two states are identical. Drift breaks that assumption silently.
Drift accumulates through several well-understood patterns:
- "Quick Fixes": An engineer manually opens port 22 in a Security Group to debug a production issue and forgets to close it. The fix works, the ticket closes, and the open port lives on indefinitely.
- Emergency Patches: A database parameter is tweaked directly in the AWS Console during a 3 AM outage. The incident is resolved, but the change never makes it back into Terraform code.
- Automation Side-Effects: A third-party tool (a monitoring agent, a cloud provider's auto-scaling policy) modifies a resource attribute that Terraform also manages.
- Malicious Actors: An attacker with console access adds a backdoor IAM user or modifies a network ACL. The change is invisible unless you're actively comparing state.
Drift remediation is the disciplined process of detecting these divergences and reconciling the two states — either by reverting the infrastructure to match the code, or by updating the code to legitimately capture the change.
The Two Paths After Detection
Once drift is detected (typically via a scheduled terraform plan run, as covered in the drift detection topic), every incident resolves into one of two decisions:
- Revert: The manual change is unauthorized, incorrect, or dangerous. The code is right; reality is wrong. Fix reality to match the code by running
terraform apply. - Import/Adopt: The manual change was legitimate — an emergency fix, a business requirement, or a discovered gap in the original design. Reality is right; the code is incomplete. Update the code to capture the change, then apply to reconcile state.
The decision between these two paths is never purely technical. It requires human judgment about why the drift occurred. This is why the remediation workflow matters as much as the detection mechanism.