The "Run Everything" Problem
In a large codebase, you might have 50,000 tests. Running all of them for a one-line CSS change wastes time and compute. Worse, it maximizes your exposure to flaky tests in unrelated parts of the system — tests that have nothing to do with your change but might fail anyway, blocking your deployment.
How TIA Works
Test Impact Analysis (TIA) is an intelligence layer that sits between your code change and your test runner. It answers the question: "Given what changed, which tests could possibly be affected?"
Step 1 — Build a Dependency Map
TIA instruments your code (via coverage tools, static analysis, or both) to build a graph: "Function A in user.js is called by LoginController, which is exercised by LoginTest.js."
Step 2 — Identify Changed Files
On a PR, TIA computes the diff: git diff main...HEAD --name-only
Step 3 — Select Relevant Tests
TIA walks the dependency graph from the changed files and selects only the tests that transitively depend on those files. Everything else is skipped.
Step 4 — Run and Report
The selected subset runs. If all pass, the build is green. The full suite still runs on a schedule (nightly) to catch any gaps in the dependency map.
TIA in Practice
Tools that implement TIA include:
- Jest (JavaScript): --changedSince flag uses git history to select affected tests.
- pytest-testmon (Python): Tracks which tests cover which lines and re-runs only affected tests.
- Bazel / Buck: Build systems with hermetic dependency graphs that enable precise test selection at scale.
- Microsoft Test Impact Analysis: Built into Azure DevOps pipelines.
pytest-testmon is a representative example of how this looks in practice: its first run builds a dependency database mapping which tests cover which lines of code, and every subsequent run consults that database to select only the tests affected by what actually changed — turning a 45,000-test suite into a handful of selected tests for a small, targeted change.
Key Point: TIA is not a replacement for running the full suite — it is an optimization for the inner development loop. The full suite must still run periodically (on merge to main, or nightly) to catch tests whose dependency map was incomplete. TIA's accuracy depends on the quality of its instrumentation; a test that uses dynamic dispatch or reflection may not be correctly mapped, leading to false negatives (a broken test that TIA incorrectly skips).