The Big Picture
Think of a Jepsen test like a bank heist movie where the "heist" is trying to catch the database lying. You have a crew of workers (clients) doing normal transactions, a saboteur (the nemesis) cutting power and jamming radios, and a forensic accountant (the checker) who reviews every transaction log afterward to find the inconsistency.
A Jepsen test consists of a Control Node that orchestrates a cluster of DB Nodes (typically 5 nodes). The test runs for a fixed duration—often 10 to 30 minutes—during which the system is simultaneously serving traffic and being attacked.
The Four Components
1. Generator
The Generator produces an infinite stream of operations to perform against the database. These are simple, verifiable operations: write(key, value), read(key), or cas(key, old-value, new-value) (compare-and-swap). The simplicity is intentional—complex operations are harder to reason about formally.
2. Clients
Multiple concurrent client threads execute the generator's operations against the DB nodes and record every result in a History log. Each operation gets one of three outcomes:
- ok — the database confirmed success
- fail — the database confirmed failure (e.g., a CAS mismatch)
- info — the outcome is unknown (the network timed out before a response arrived)
The info state is critical. It means the operation may or may not have succeeded. The checker must handle this ambiguity correctly.
3. The Nemesis
A special process that runs concurrently with the clients and injects faults into the cluster. Covered in detail in the next section.
4. The Checker
After the test finishes, the Checker takes the complete History log and applies formal logic to determine whether the sequence of operations is consistent with the database's claimed consistency model. If it finds a violation, it produces a counterexample—a specific sequence of operations that proves the database lied.
Key Point: The info (unknown) outcome is what makes Jepsen analysis hard and interesting. A naive checker that ignores unknown outcomes will miss real violations. Knossos handles them correctly by treating an info operation as either having succeeded or failed, and checking both possibilities.