Distributed Systems & Algorithms at Scale

Introduction: What is Software Architecture?

The Monolith: All Eggs in One Basket

The Modular Monolith: The Organized Middle Ground

Microservices: Divide and Conquer

A Note on SOA (Service-Oriented Architecture)

Communication: Event-Driven Architecture (EDA)

Coordination: Orchestration vs. Choreography

Introduction: What Time Is It, Really?

Physical Clocks: Drift and Synchronization

PTP: When Milliseconds Aren't Good Enough

A Quirky Problem: Leap Seconds

Monotonic Clocks vs. Wall Clocks

The Need for Logical Time

Lamport Timestamps: The Simple Counter

Vector Clocks: Detecting Concurrency

Google TrueTime: The Hybrid Solution

Introduction: Driving the System

The Feedback Loop

The PID Controller: Past, Present, and Future

Applying Control Theory: Autoscaling

The Thundering Herd Problem

Backpressure: Don't Let the Sink Overflow

Introduction: The Memory Wall

The Magic Sauce: Hash Functions

Bloom Filters: The "Membership" Question

Cuckoo Filters: A Better Bloom Filter?

HyperLogLog: The "How Many Uniques" Question

Count-Min Sketch: The "How Often?" Question

Combining Structures: The Top-K Pattern

Introduction: The Communication Gap

Ubiquitous Language: Speaking the Same Tongue

Bounded Contexts: The Big Picture

Tactical DDD: Entities vs. Value Objects

Aggregates: Consistency Boundaries

Domain Events: Communicating Between Aggregates

The Anti-Corruption Layer (ACL)

Repositories: Abstracting Persistence

Introduction: Embrace the Failure

The Retry Pattern: "Try, Try Again

Smarter Retries: Backoff and Jitter

The Circuit Breaker: Stopping the Bleeding

Circuit Breaker States

Bulkheads: Contain the Failure

Fallbacks and Rate Limiting

Timeouts: The Forgotten Pattern

Introduction: The Problem of Agreement

The FLP Impossibility and Why It Matters

The Core Concept: Quorums

Paxos: The Pioneer

Raft: Consensus for Humans

Raft Concept 1: Leader Election

Raft Concept 2: Log Replication

Zab (ZooKeeper Atomic Broadcast)

The Danger: Split-Brain Scenarios

Practical Considerations: Timeouts, Performance, and Membership Changes

Introduction: The Illusion of a Single Copy

Strong Consistency: Linearizability

Sequential Consistency: Order Matters, Time Doesn't

Eventual Consistency: The Optimist's Choice

Causal Consistency: The Middle Ground

Client-Centric Consistency Models

Introduction: The Problem with Splitting Databases

The Old Way: Two-Phase Commit (2PC)

A Note on Three-Phase Commit (3PC)

The Modern Way: The Saga Pattern

Saga Compensation: The "Undo" Button

Coordinating Sagas: Choreography vs. Orchestration

TCC (Try-Confirm-Cancel)

Introduction: Testing is Not Enough

What is TLA+?

PlusCal: A Gentle Bridge to the Math

The Model Checker (TLC): The Ultimate Bug Hunter

What TLA+ Finds in Practice

Introduction: Why Do Lines Form?

Little's Law: The Golden Rule

Utilization: How Busy Are We?

Kingman's Formula: The Hockey Stick Curve

The Universal Scalability Law (USL)

Introduction: Stable, but Dead

The Physics of Failure

Mechanism 1: The Retry Storm

Mechanism 2: The Cache Stampede (Dogpile)

Mitigation: Breaking the Loop

Probabilistic Early Expiration: Preventing the Cache Stampede

Introduction: The Problem with Layers

Defining the Cell

Blast Radius Mathematics

Shuffle Sharding: Magic with Math

Cell Lifecycle and Operations

Introduction: The "Multiplayer" Problem

The Secret Sauce: Commutativity, Associativity, and Idempotency

The LWW-Element-Set (Last-Write-Wins)

The OR-Set (Observed-Remove Set)

The G-Counter and PN-Counter

State-Based vs. Operation-Based CRDTs

Introduction: The Heavy Crown

The Physics of Split Brain

Fencing Tokens: The Zombie Defense

Lease-Based Election and Clock Skew

Introduction: How Do You Talk to 10,000 Nodes?

Epidemic Propagation: The Math of Rumors

SWIM: A Better Way to Detect Failures

SWIM Refinement: The Suspicion Mechanism

Memberlist: Gossip in the Real World

The Piggyback Strategy: Getting Information for Free

Introduction: Why Do We Need Locks?

The Lease Mechanism

Redlock: Distributed Locking with Redis

The Danger of Clock Drift

Consensus-Based Locking: ZooKeeper and etcd

Introduction: When Autoscaling is Too Slow

Goodput vs. Throughput

Priority Queues: Who Gets Saved?

Admission Control: The Bouncer

Introduction: The Amnesiac Database

CQRS: Splitting the Brain

The Mechanics of Event Sourcing

The Superpower: Time Travel Debugging

When NOT to Use Event Sourcing

Introduction: The Nightmare of Shared State

Anatomy of an Actor

The "Let It Crash" Philosophy

Supervisor Hierarchies

Flavors of Actors: Akka vs. Orleans

Introduction: The Integrity Problem

Anatomy of a Merkle Tree

Efficient Verification: Merkle Proofs

Anti-Entropy: Finding the Difference Between Replicas

Immutable Ledgers & Blockchains

Introduction: The Reshuffling Nightmare

The Hash Ring

Adding and Removing Nodes

The Problem of Hotspots and Data Skew

Virtual Nodes (VNodes): The Solution to Skew

Real-World Implementations

Implementation Deep Dive: Data Structures and Complexity

Introduction: The Bouncer at the Door

The Leaky Bucket: Smooth and Steady

The Token Bucket: Bursts Allowed

Leaky vs. Token: The Comparison

Introduction: Marketing vs. Reality

How Jepsen Works: The Architecture

The Nemesis: Chaos on Demand

The Analysis: Linearizability Checking

Real-World Findings: What Jepsen Actually Discovered