The "Last Updated: 2019" Tragedy
Trust is hard to gain and easy to lose. If a developer follows a runbook and the command fails because the flag was renamed six months ago, they will not trust your documentation again — and they will tell their teammates. One stale page can undermine an entire documentation program.
The root cause is that documentation maintenance is invisible. Nobody gets paged when a doc goes stale. Nobody's on-call rotation includes "verify the API reference." Without automated enforcement, staleness is guaranteed.
Automated Freshness Testing
Documentation Engineering treats freshness as a measurable, enforceable property — not a cultural aspiration.
1. Broken Link Checker
A CI tool crawls every link in your documentation on every build. If any internal or external link returns a non-2xx HTTP status, the build fails. This is the most basic and highest-value freshness check — a single step in the docs pipeline (tools like lychee are popular for this) that fails the build on any dead link.
2. Code Snippet Testing
Tools extract fenced code blocks from Markdown and execute them in a sandboxed environment. If the code example fails to run, the documentation build fails. This is the highest-fidelity check — it proves the example actually works.
For Python documentation, pytest with the --doctest-glob flag can test code blocks directly. For shell commands, a custom CI step can extract the fenced bash blocks from a doc page and run each one inside a throwaway container.
3. The Staleness Bot
A scheduled CI job (running weekly) checks the git log of every documentation file. Any file not touched in the configured threshold (commonly 90 days for runbooks, 180 days for architecture docs) triggers an automated action: the job walks every file under docs/, looks up its most recent commit timestamp, and flags any file older than the threshold.
The bot reads a owner: field from each file's YAML frontmatter and opens a GitHub Issue assigned to that person: "This runbook hasn't been reviewed in 90 days. Please verify it is still accurate and update the last_reviewed date."
4. The Freshness Dashboard
Platform teams often build a simple dashboard that visualizes documentation health across the entire corpus:
| Metric | Target | Current |
|---|---|---|
| Broken links | 0 | 0 ✅ |
| Files stale > 90 days | < 5% | 3.2% ✅ |
| Files stale > 180 days | 0 | 2 ⚠️ |
| Code snippets passing | 100% | 94% ❌ |
This moves documentation maintenance from "when we remember" to "automated discipline with visible accountability."
A common platform engineering interview question is: "How do you measure documentation quality?" Most candidates answer with subjective metrics like "developer satisfaction surveys." A stronger answer includes objective, automated metrics: broken link rate (measured in CI), code snippet pass rate, staleness percentage (files not updated in N days), and search query zero-result rate (what users searched for and found nothing). These are the metrics that make documentation quality a first-class engineering concern.