Free — no signup required

Root Causes of Flakiness: Know Your Enemy

2 min read

Why Tests Lie

Before you can fix flakiness, you need to understand where it comes from. The vast majority of flaky tests fall into a small number of categories.

1. Race Conditions
The test assumes that operation A completes before operation B begins, but that ordering is not guaranteed. A UI test that clicks "Save" and immediately checks for a success banner is racing against the network. Sometimes it wins, sometimes it doesn't.

2. Shared Mutable State
Tests that share a database, a file, a global variable, or an in-memory cache can interfere with each other. Test A creates a user with ID 42. Test B deletes all users. If they run in parallel — or even in the wrong order — one of them will fail for reasons unrelated to the code under test.

3. Time and Date Dependencies
A test that asserts created_at == today will fail at midnight, in a different timezone, or when the CI runner's clock drifts. Tests that use sleep(2) to wait for an async operation are encoding a timing assumption that will eventually be violated.

4. External Dependencies
Tests that make real HTTP calls to third-party APIs, real database connections, or real filesystem paths will fail whenever those external systems are slow, unavailable, or return unexpected data. This is why mocking and test doubles exist.

5. Test Order Dependency
Test B only passes if Test A ran first and left some state behind. This is a design flaw: every test should be able to run in isolation, in any order, and produce the same result.

6. Resource Exhaustion
On a heavily loaded CI runner, a test that expects a response within 500ms might time out simply because the CPU was busy. This is infrastructure flakiness, not code flakiness — but it still breaks your build.

Key Point: Identifying the root cause category before attempting a fix saves enormous time. A race condition is fixed with explicit waits or event-driven assertions. Shared state is fixed with test isolation (setup/teardown or transaction rollback). External dependencies are fixed with mocks or contract tests. Applying the wrong fix — like adding a sleep() to a shared-state problem — creates a test that is slower and still flaky.

This is one of 18 chapters

Every chapter is free — Kubernetes, Terraform, SRE, distributed systems, and more — with fast daily review built in.

Sign up free