The End of Certainty
Think about a shared Google Doc. If you and a colleague are both editing it at the same time with a bad internet connection, you might see different versions of the document for a few seconds. Google has to figure out how to merge your changes when the connection is restored. This is not a bug — it is a fundamental consequence of having data in more than one place.
In the previous topic on NoSQL, we learned that to handle massive scale, we often distribute data across multiple machines called nodes. When you have data on just one machine, life is simple. If you write X = 5, the next read will see X = 5. There is only one source of truth.
When you have copies of that data on three different machines connected by a network, life gets complicated. What if you write X = 5 to Node A, but the network link to Node B is broken? Node B still thinks X is whatever it was before. Now a client asks Node B for the value of X. What should Node B do?
- Return old data? The system stays up, but the client gets a wrong answer. This sacrifices Consistency for Availability.
- Refuse to answer? The client gets an error, but at least no one gets wrong data. This sacrifices Availability for Consistency.
This fundamental dilemma is the core of distributed data engineering. You can no longer have it all, and every database system you will ever operate has already made a choice about which side of this trade-off it favors.