System Design

Strong vs Eventual Consistency: What I Got Wrong at Scale

For years I believed strong consistency was a luxury you gave up the moment you went global. Then I started working on a system that proved me wrong.

By Admin2026-06-1811 min readconsistencydistributed-systemseventual-consistencyreplicationspannersystem-design

For most of my career I held a belief that felt like common sense: if you want a system to run across the whole planet, you have to give up strong consistency. Global scale meant eventual consistency. That was just the tax. Everyone I respected seemed to operate that way, so I did too.

Then I started working on cross-region replication for a database whose entire reason to exist is providing strong, globally-consistent reads at exactly that scale. It quietly demolished my assumption. Not because the trade-off isn't real - it absolutely is - but because I'd been treating a spectrum as a wall. This is the explanation that would have saved me years of fuzzy thinking.

The two words, in plain language

When you write data to a system that keeps multiple copies (replicas) across multiple machines, there's a gap between "the write landed on one replica" and "every replica knows about it." How a system handles that gap is its consistency model.

• Strong consistency - the moment a write succeeds, every subsequent read, from anywhere, sees that new value. The system behaves as if there's a single, up-to-date copy of the data, even though there are many.
• Eventual consistency - after a write, different replicas may briefly disagree. A read might return the old value for a little while. Given enough time with no new writes, all replicas converge to the same value. Hence "eventual."

You feel the difference constantly without naming it. Post a comment and have it vanish on refresh, then reappear a few seconds later? That's eventual consistency - your write reached one replica, your refresh hit a different one that hadn't caught up. Mildly annoying for a comment. Catastrophic for a bank balance.

Why eventual consistency exists at all

The root cause is physics: replication takes time. When you write to a replica in Mumbai and you have another in Virginia, the update has to travel ~12,000 km. Even at the speed of light through fiber, that's tens of milliseconds, and that's before queuing, retries, and congestion. During that window, the Virginia replica is stale. It has no choice.

A system can respond to this in one of two ways. It can wait - refuse to confirm the write until enough replicas have acknowledged it, so reads are always current (strong, but slower, and more fragile under failure). Or it can not wait - confirm the write immediately on the local replica and propagate in the background (fast and resilient, but reads can be stale). Eventual consistency is what you get when you choose not to wait. It's not laziness; it's a deliberate trade of correctness-on-read for speed and availability.

sequenceDiagram
    participant U as User
    participant L as Leader (Mumbai)
    participant F as Follower (Virginia)
    U->>L: write balance = 100
    L-->>U: OK (confirmed immediately)
    Note over L,F: replication in flight (~tens of ms)
    U->>F: read balance
    F-->>U: 50  ← STALE (write hasn't arrived)
    L->>F: replicate balance = 100
    U->>F: read balance (moments later)
    F-->>U: 100 ← now converged

This sequence diagram shows eventual consistency. A user writes balance equals 100 to the leader replica in Mumbai, which confirms immediately. Replication to the follower in Virginia is still in flight. If the user reads from the Virginia follower during this window, they get the stale value 50, because the new write hasn't arrived yet. Once replication completes, a later read from the same follower returns the converged value 100.

Here's that stale-read window made concrete:

class Replica:
    def __init__(self): self.store = {}
    def write(self, k, v): self.store[k] = v
    def read(self, k): return self.store.get(k, "<empty>")

leader, follower = Replica(), Replica()

leader.write("balance", 100)            # write lands on the leader
# replication to the follower hasn't happened yet:
print("Eventual read (follower):", follower.read("balance"))   # <empty>  ← stale!

follower.write("balance", 100)          # replication catches up
print("After replication      :", follower.read("balance"))    # 100

# Strong consistency = always serve from an up-to-date source:
print("Strong read (leader)   :", leader.read("balance"))      # 100, always

The follower returns a stale answer in the gap before replication arrives. A strongly-consistent design closes that gap - by reading from the leader, or from a quorum of replicas - at the cost of speed and availability.

It's a spectrum, not a switch

This is the insight that broke my old "global means eventual" wall. Between "always perfectly current" and "anything goes" sits a ladder of useful guarantees, and good systems pick the rung they actually need:

• Read-your-own-writes - you always see your own updates immediately, even if others see them slightly later. (Why your own comment never disappears, even on an eventually-consistent platform - they pin you to an up-to-date replica.)
• Monotonic reads - once you've seen a value, you'll never see an older one. Time only moves forward for you.
• Causal consistency - operations that depend on each other are seen by everyone in the right order. A reply never shows up before the message it replies to.

Most "eventually consistent" systems you admire are actually offering one of these stronger-than-pure-eventual guarantees. Knowing the rungs by name is the difference between "eventual is fine here" and "I need read-your-own-writes here, full strong consistency there, and pure eventual is acceptable for this third thing."

Tip: When someone says a system is "eventually consistent," the sharp follow-up is: "eventual with what guarantees?" Pure eventual consistency is rare in practice. Read-your-writes and monotonic reads are what most real systems quietly provide on top of it.

The hard part: making strong consistency global

So how does a system give strong consistency across continents without the latency making it unusable? This was the part that genuinely changed how I think.

The naive approach is to coordinate every write through a single global leader or a consensus round - correct, but you're paying that cross-ocean round-trip on every operation. The breakthrough in systems like Google Spanner is to attack the hardest part of distributed agreement: knowing what happened before what. Spanner uses tightly-synchronized clocks (backed by GPS and atomic clocks in the datacenters) to put a small, bounded uncertainty on every timestamp. Because the system knows the maximum error in its clocks, it can briefly wait out that uncertainty and then safely order transactions globally - delivering external consistency at planet scale.

You don't need the internals for an interview. You need the takeaway: strong consistency at global scale is achievable, but it is expensive - in latency, in infrastructure, in engineering. It's not free, and that cost is exactly why eventual consistency still wins for the right workloads.

Choosing the right one (this is the actual skill)

The pattern across that table: choose strong when reading wrong data causes real harm, and choose eventual when staleness is cheap and you'd rather have speed and resilience. And - the senior move - make this choice per piece of data, not once for the whole system. The same app can keep payments strong and likes eventual. That's not indecision; that's design.

The belief I started with - "global means eventual" - wasn't exactly wrong, it was just unexamined. The real picture is a spectrum, and your job as a designer is to put each piece of data on the right rung and be able to say why. My co-founder Brandon comes at the same trade-off from the failure side in his piece on the CAP theorem; read the two together and the whole consistency landscape clicks into place. We map all of it, rung by rung, in System Design Fundamentals.

Go deeper

• Consistency model - the full ladder from strong to eventual, with the guarantees in between.
• Eventual consistency - the model, its trade-offs, and where it's the right call.
• Google Cloud Spanner - how strong, externally-consistent reads are delivered at global scale, and what it takes.
• Apache Cassandra - tunable consistency in practice: choosing per-query how strong you need to be.

More articles