Advanced16 min

Data Inconsistency

Copies of the same data disagree — from replication lag, dual writes, or conflicting updates.

By the end of this lesson, identify sources of divergence across data stores and choose a reconciliation strategy.

How deep?

How the pieces actually move.

Once the same fact lives in more than one place — a replica, a cache, a search index, another service's database — those copies can disagree. Causes include replication lag (a read hits a replica that hasn't caught up), dual writes (writing to a DB and a cache/queue separately, one succeeding and one failing), and concurrent conflicting updates.

First, the whole system

First, see the whole system. Then we’ll open it up.

The dual-write trap

An app writes to the database successfully but the separate write to the cache/index fails, leaving them inconsistent.

flowchart LR
  app[App] -->|write 1 ok| db[(Database)]
  app -->|write 2 fails| cache[(Cache / index)]
  db --> mismatch[DB and cache disagree]
  cache --> mismatch

Strategies: avoid dual writes with the outbox pattern (write data and an event in one transaction; publish the event reliably from the outbox); accept eventual consistency and design reads for it; use read-your-writes where users must see their own changes; add CRDTs or version vectors for conflict resolution; and run reconciliation jobs to detect and repair drift.

Abstraction leak

This is where the abstraction starts leaking.

'Just update the cache after the DB write' is a dual write, and it will drift the first time the process crashes between the two. Consistency across stores is not free bookkeeping — it needs a transactional boundary or a reconciliation loop.

Data inconsistency

Trigger: Replication lag, dual writes, or concurrent conflicting updates across stores.
Symptom: Different components report different values for the same fact.
Blast radius: Any feature that reads the diverged copies; corrupted downstream decisions.
Mitigation: Outbox/CDC over dual writes, defined consistency model, reconciliation jobs, conflict resolution.

How it connects

Why is writing to the database and then to a cache in two steps risky?

It's a dual write with no shared transaction; a crash between the steps leaves them inconsistent. The outbox pattern or CDC avoids it.