Free — no signup required

Introduction: The Heavy Crown

2 min read

Why We Elect Leaders

In distributed systems, coordinating work across many machines is hard. If every node independently decides what to do, you get conflicts: two nodes might try to write different values to the same record, or two workers might process the same job twice. The simplest solution is to designate a single node as the Leader (also called Master or Primary) to act as the single source of authority.

The leader's responsibilities typically include:

  • Write ordering: All write requests route through the leader, which serializes them into a single, consistent log.
  • Task assignment: The leader distributes work to follower nodes, preventing duplicate processing.
  • Coordination: The leader makes decisions that require global knowledge of the cluster state.

The Danger

The moment you introduce a single authority, you introduce a single point of failure. When the leader becomes unavailable — due to a crash, a network partition, or even a long pause — the cluster must elect a replacement. This transition is where things go wrong.

The core danger is this: what if the old leader isn't actually dead? What if it was just slow — stuck in a garbage collection pause, or temporarily cut off from the network? If the cluster elects a new leader while the old one is still running, you now have two nodes that both believe they are the leader. Both accept writes. Both issue commands. Both modify shared state. This is the Split-Brain problem, and it leads directly to data corruption that can be impossible to repair.

The rest of this lesson is about the mechanisms engineers use to prevent split-brain and contain the damage when prevention fails.

Interview Tip

Interviewers frequently ask: "What is split-brain and how do you prevent it?" A weak answer names quorums and stops there. A strong answer explains the layered defense: quorums prevent a minority partition from electing a leader, fencing tokens prevent a zombie leader from corrupting storage even after quorum is lost, and lease-based elections add a time-bounded safety window. Knowing all three layers — and why each is necessary — signals senior-level understanding.

This is one of 18 chapters

Every chapter is free — Kubernetes, Terraform, SRE, distributed systems, and more — with fast daily review built in.

Sign up free