Free — no signup required

Drift Detection: Combatting "ClickOps

2 min read

The Threat of "ClickOps"

"ClickOps" is when a user logs into a web console (AWS, GitHub, Azure) and manually changes a setting by clicking around — bypassing the code entirely. It is the enemy of Governance as Code.

A realistic scenario: A developer's CI build is failing at 4 PM on a Friday before a release. They go into GitHub Settings, disable "Require CI to pass before merging," force-merge their code, and fully intend to re-enable it afterward. They forget. The setting stays off for three weeks. Two other developers merge broken code during that window.

This is not a hypothetical. It happens constantly in organizations that rely on manual configuration.

Detective Controls: The Scheduled Plan

Governance as Code isn't just about setting things up correctly; it's about keeping them correct over time. Your CI/CD system should run a Scheduled Terraform Plan on a regular cadence — every hour for critical infrastructure, nightly for lower-risk resources.

The scheduled job works like this:

  1. Refresh: Terraform queries the actual live state of every managed resource via the provider API.
  2. Compare: It diffs the live state against the desired state in the code.
  3. Alert or Heal: If drift is detected, it either opens a ticket/alert for the Platform Team, or — in more aggressive setups — automatically runs terraform apply to revert the manual change.

Choosing Between Alert and Auto-Heal

Auto-healing is powerful but requires careful judgment. For security-critical settings (branch protection, S3 bucket ACLs, IAM policies), auto-healing is usually the right call — the risk of leaving the drift in place outweighs the risk of an unexpected automated change. For settings that might legitimately need temporary manual overrides (e.g., scaling parameters during an incident), alerting and requiring human confirmation is safer.

The decision should be documented as policy itself — which resources are auto-healed, which are alert-only, and who is notified in each case.

This is one of 18 chapters

Every chapter is free — Kubernetes, Terraform, SRE, distributed systems, and more — with fast daily review built in.

Sign up free