Free — no signup required

Bit Rot and Checksums

2 min read

The Silent Killer

Journaling and CoW protect against crashes. They do not protect against a different threat: bit rot.

Bit rot is the spontaneous corruption of data at rest. A cosmic ray strikes a memory cell. A magnetic domain on a spinning disk decays. A flash cell on an SSD loses charge over time. The result is one or more bits flipping from 0 to 1 or vice versa — silently, without any write operation occurring.

On ext4 or XFS, this is undetectable. The filesystem reads the corrupted block, computes no integrity check, and serves the garbage bytes to your application. The application may crash, produce wrong results, or silently store corrupt data — and you may not discover the problem for months.

Checksums: Detecting Corruption

Btrfs and ZFS store an integrity checksum (by default, CRC32C on Btrfs; Fletcher-4 or SHA-256 on ZFS) for every data block and every metadata block. The checksum is stored separately from the data it protects.

When a block is read:
1. The filesystem reads the data block.
2. It computes the checksum of what it read.
3. It compares the computed checksum to the stored checksum.
4. If they match: data is served to the application.
5. If they do not match: corruption detected.

Self-Healing with Redundancy

Detection alone is useful — it prevents serving corrupt data. But ZFS goes further with self-healing when storage redundancy exists (mirroring or RAID-Z):

  1. Checksum mismatch detected on Drive 1.
  2. ZFS reads the same block from Drive 2 (the mirror).
  3. Drive 2's block matches its checksum — it is correct.
  4. ZFS writes the correct data back to Drive 1, repairing the corruption.
  5. The application receives the correct data. The entire process is transparent.

This is called scrubbing when done proactively (reading all blocks to verify checksums) rather than reactively (on read). ZFS administrators typically schedule a scrub monthly as a background job against a whole storage pool, which walks every block, reports how many mismatches it found, and — where redundancy exists — how many it was able to repair on the spot.

Key Point: The combination of CoW + checksums + redundancy in ZFS creates a storage system where silent data corruption is effectively impossible to sustain undetected. This is why ZFS is the default filesystem for FreeBSD, TrueNAS, and many production NAS/SAN systems. The trade-off is RAM usage — ZFS's ARC (Adaptive Replacement Cache) is memory-hungry, and its metadata structures require more RAM than ext4 for the same dataset size. The rule of thumb is 1 GB of RAM per 1 TB of storage, with a minimum of 8 GB.

This is one of 18 chapters

Every chapter is free — Kubernetes, Terraform, SRE, distributed systems, and more — with fast daily review built in.

Sign up free