What Drift Looks Like in Practice
Configuration drift is when a server's actual state diverges from its intended state. It happens in predictable ways:
- Manual hotfixes: An engineer SSHes into a production server at 2 AM to fix an outage and changes a config file. The fix works. They forget to update the CM code. Now the CM code no longer reflects reality.
- Package updates: A security team pushes an OS patch that changes a config file as a side effect.
- Partial failures: A CM run fails halfway through. 30 of 50 servers get the new config; 20 don't.
- Snowflake servers: Over months of incremental changes,
web-01andweb-02accumulate different sets of packages, config values, and running processes. They were identical at birth; now they behave differently under load.
How CM Tools Detect Drift
Both Puppet and Chef maintain a resource catalog — a complete description of every managed resource on a node. At each check-in, the agent compares the catalog against the live system state. Any discrepancy is a drift event.
Chef calls this a convergence run. Puppet calls it a catalog apply. Both produce a report showing exactly which resources were out of compliance and what was changed.
# Example Chef convergence run output (simplified)
Recipe: nginx::default
* package[nginx] action install (up to date)
* template[/etc/nginx/nginx.conf] action create
- update content in file /etc/nginx/nginx.conf from abc123 to def456
--- /etc/nginx/nginx.conf 2024-01-15 09:00:00
+++ /etc/nginx/nginx.conf 2024-01-15 09:30:00
@@ -10,7 +10,7 @@
- worker_processes 2;
+ worker_processes 4;
* service[nginx] action restart
- restart service service[nginx]
This output tells you exactly what changed, when, and from what value to what value — a complete audit trail.
Key Point: Drift detection is only as useful as your response to it. CM tools that silently correct drift can mask underlying problems — if the same file keeps getting changed back, something is wrong with either the CM code or the process. Mature CM implementations configure alerting on drift events rather than silently correcting them, so the root cause can be investigated.