System Design
How Databases Guarantee Durability After a Crash
You commit a transaction, the database says 'done' - and a half-second later the power dies. When it comes back, your data is still there. That guarantee is harder to deliver than it sounds. Here's the mechanism that makes it work.
Here's a guarantee we take completely for granted: you commit a transaction, the database confirms it's done, and from that moment the data is permanent. The power can die, the process can crash, the machine can reboot - and when everything comes back, your committed data is still there, intact. This is durability, the "D" in ACID, and it's one of those properties that sounds obvious until you think about what it actually takes to deliver. Delivering it reliably, in the face of a crash that can strike at any instant, is genuinely hard.
I work on systems where durability isn't optional and the failure modes are unforgiving, so this is a property I think about carefully. Let me walk through why durability is hard, and the elegant mechanism - write-ahead logging - that databases use to guarantee it.
Why durability is hard
The naive idea of durability is "just write the data to disk." But that hides two serious problems.
Problem one: a crash can strike mid-write. Suppose a transaction needs to update several pieces of data on disk. Writing them isn't instantaneous - it takes time, and during that time a crash can happen. If the crash strikes after you've written some of the changes but not all, you're left with the data half-updated: a corrupted, inconsistent state. The write wasn't atomic, and the crash exposed that.
Problem two: "written to disk" is a lie, often. When a program writes data, it usually doesn't go straight to the physical disk - it goes into operating-system buffers in memory, which the OS flushes to disk later for efficiency. So the database can think data is "written" when it's actually still sitting in volatile memory, not yet on durable storage. A crash at that moment loses it. To truly guarantee durability, the database must force the data all the way to physical disk and confirm it's there - an operation generally called fsync - before considering anything durable. And forcing to disk is slow, which creates tension with performance.
So durability isn't "write to disk" - it's "survive a crash at any moment, including mid-write, given that writing is slow and 'written' doesn't always mean 'on disk.'" That's the real problem.
The core technique: write-ahead logging
The mechanism that solves this is Write-Ahead Logging (WAL), and the central idea is captured in the name: write the change to a log before you apply it to the actual data.
Here's how it works. Before modifying the real data files, the database first appends a record describing the change to a log - a sequential file on disk. This log record says, in effect, "this transaction is making this change." The database forces that log record to disk (the fsync that guarantees it's truly durable). Only after the log record is safely on disk does the transaction count as committed and the database tell the client "done." The actual data files can be updated afterward, at the database's leisure.
The genius of this is what happens on a crash. If the database crashes, on restart it reads the log and replays it - reapplying every change that the log says was committed but that might not have made it into the data files yet. Because the log is the authoritative record of what was committed, replaying it restores the database to a correct, consistent state with all committed transactions intact. The log is the durability guarantee.
sequenceDiagram
participant C as Client
participant DB as Database
participant LOG as Write-Ahead Log (disk)
participant DATA as Data files (disk)
C->>DB: COMMIT transaction
DB->>LOG: Append change record
LOG-->>DB: Forced to disk (fsync)
DB-->>C: "Done" (now durable)
Note over DB,DATA: Data files updated later
DB->>DATA: Apply change (eventually)
This sequence diagram shows write-ahead logging. When the client commits a transaction, the database first appends a record of the change to the write-ahead log on disk, and forces it to disk with an fsync so it is truly durable. Only then does the database tell the client the commit is done - at which point the change is guaranteed durable. The actual data files are updated later, at the database's convenience. If a crash happens, the log is replayed on restart to recover all committed changes.
Why a log? The performance insight
A natural question: why is writing to a log faster or better than just writing the data? The answer is a beautiful piece of systems insight: sequential writes are far faster than random writes.
Updating the actual data files often means writing to scattered locations across the disk (different rows, different pages, different indexes) - many small, random writes, which are slow. The log, by contrast, is append-only and sequential: every change is just tacked onto the end of the log file, one after another. Sequential writing to the end of a file is dramatically faster than scattered random writes. So the database does the fast thing (append to the sequential log) to achieve durability immediately, and defers the slow thing (updating the scattered data files) to do later, in the background, batched efficiently.
This is why WAL gives you durability and good performance: commits are fast because they only need a quick sequential log append (plus the fsync), not a slow scattered update of all the data files. The data files catch up afterward, off the critical path. You get the durability guarantee at the speed of a sequential write.
Recovery: redo and (sometimes) undo
When the database restarts after a crash, recovery uses the log to restore correctness, in two directions:
- Redo: reapply committed changes that the log records but that may not have reached the data files. This ensures every committed transaction's effects are present - nothing acknowledged to a client is lost.
- Undo: roll back changes from transactions that were in progress but not committed when the crash hit. A transaction that was halfway done and never committed must leave no trace, so its partial changes are reversed. (Whether undo is needed depends on the database's specific design, but the principle - uncommitted work must be erased - is universal.)
Together, redo and undo bring the database to a clean state: every committed transaction fully applied, every uncommitted one fully gone. That's exactly the atomicity-and-durability guarantee ACID promises, delivered through the log.
Checkpoints: keeping the log bounded
One problem remains: if the log records every change forever, it grows without bound, and recovery would have to replay an enormous history. The solution is checkpoints. Periodically, the database flushes all the current in-memory changes to the actual data files and records a checkpoint in the log. After a checkpoint, the database knows everything before that point is safely in the data files, so on recovery it only needs to replay the log from the last checkpoint forward - not from the beginning of time. Checkpoints bound both the log size and the recovery time, keeping the whole scheme practical.
A worked crash scenario
Let me trace a crash to show recovery in action. Suppose three transactions are in flight. Transaction 1 committed a minute ago - its log record is on disk and its changes are already in the data files. Transaction 2 committed a second ago - its log record is on disk, but its changes hadn't yet been written to the data files when the crash hit. Transaction 3 was halfway through, never committed, when the crash struck - but some of its changes had already leaked into the data files.
Now the power dies and the database restarts. Recovery reads the log from the last checkpoint forward. For Transaction 2, the log shows it committed, but the data files are missing its changes - so recovery redoes them, applying the changes the log records. Transaction 2 is fully restored; nothing the client was promised is lost. For Transaction 3, the log shows it never committed, yet some of its changes reached the data files - so recovery undoes them, erasing the partial work. Transaction 3 vanishes as if it never ran. Transaction 1 needs nothing (already fully durable).
The result: every committed transaction (1 and 2) is fully present, every uncommitted one (3) is fully gone, and the database is back to a clean, correct state. That's atomicity and durability delivered together, entirely through the log. The client told "Transaction 2 succeeded" finds it did; the client whose Transaction 3 never confirmed finds no trace of it. Notice that recovery needs nothing but the log and the data files as they were at the moment of the crash - no special state had to survive, because the log already held everything required to reconstruct correctness.
The fsync trade-off and group commit
The slow part of durability is that fsync - forcing the log to physical disk - so it's worth understanding the tension and a key optimization. Every commit needs its log record durably on disk, which means an fsync, which is slow (it waits on the physical disk). If every single commit triggered its own fsync, high-throughput systems would be throttled by disk-flush speed.
The optimization is group commit: the database batches the log records of multiple transactions committing around the same time and flushes them to disk together in one fsync. Instead of one fsync per commit, you get one fsync for a whole batch of commits - the durability guarantee is preserved (every committed transaction's record is on disk before its commit is acknowledged), but the expensive disk flush is amortized across many transactions. This is how databases deliver durability and high commit throughput: they don't cheat on the fsync, they share its cost across many commits. It's a lovely example of keeping a hard guarantee while engineering around its cost.
Durability is a spectrum
Durability in practice has levels, because the fsync that guarantees it is costly. At the strict end, a database fsyncs every commit's log record before acknowledging - full durability, no committed data ever lost. Some systems offer relaxed modes that acknowledge a commit before the log is truly flushed (trusting it'll be flushed momentarily), trading a small window of potential data loss on a crash for higher throughput. And as we'll see, durability extends beyond one machine: replicating the log to other machines means a committed transaction survives even total loss of the primary, which is the durability level serious systems actually require. So "durable" isn't one thing - it ranges from "flushed to this machine's disk" to "replicated across machines and regions," and you choose the level the data demands. Money demands the strict, replicated end; lower-stakes data might accept a relaxed mode for speed.
The connection to replication
Everything so far is about surviving a crash on a single machine. But the same log is also the foundation of durability across machines. The write-ahead log - that ordered record of every change - is exactly what database replication streams to copies on other machines. By shipping the log to replicas, the database keeps redundant copies in sync, so that even if an entire machine is lost (not just a crash, but hardware failure), the data survives on other machines.
This is durability's higher form: not just surviving a crash on one box, but surviving the loss of an entire machine or data center by keeping replicated copies, all kept consistent through that same stream of log records. (The log is also what change data capture consumes - the same ordered change stream serves durability, replication, and CDC.)
Important: The unifying idea worth carrying away: a database's durability rests on an ordered log of changes written before the data itself. That single construct - the write-ahead log - delivers fast durable commits (sequential append plus fsync), crash recovery (replay the log), atomicity (redo committed, undo uncommitted), replication (ship the log to other machines), and even change data capture (consume the log as a stream). One elegant mechanism underpins an enormous amount of what makes databases trustworthy.
Tip: Nikhil's note from distributed systems: When you work on systems where data loss is unacceptable, you develop a healthy paranoia about the word "durable." The subtle trap is the second problem I described - "written" rarely means "on physical disk" until you've forced it there, and that force (fsync) is slow, so there's constant pressure to cut the corner and acknowledge a commit before it's truly durable. Resist it: a system that says "done" before the data is genuinely on durable storage is lying, and the lie surfaces at the worst possible moment - a crash. And at scale, single-machine durability isn't enough; you replicate the log across machines and often across regions, because a guarantee that survives a process crash but not a machine failure isn't the guarantee users actually need. Durability is a discipline, and the write-ahead log is the tool that makes it both possible and fast.
Interview: Durability and write-ahead logging come up in database and system design discussions, especially around ACID and crash safety. Being able to explain that databases write changes to a sequential log before the data files, force it to disk, and replay it on recovery - and why (sequential writes are fast, the log is the source of truth) - signals real depth. Connecting it to replication (ship the log to other machines) and noting that "written" requires an actual flush to disk shows you understand durability as the genuinely hard guarantee it is, not a given.
The bottom line
Durability - committed data surviving any crash - is hard because crashes can strike mid-write and because "written" often means "in volatile memory," not on disk. Databases solve it with write-ahead logging: before touching the actual data, they append a record of each change to a sequential log and force it to disk, counting a transaction as committed only once its log record is durable. On a crash, replaying the log restores all committed changes (redo) and erases uncommitted ones (undo). The log works because sequential appends are far faster than scattered data-file writes, giving durability at high speed; checkpoints keep the log bounded; and shipping the log to other machines extends durability into replication.
One ordered log, written ahead of the data, underpins durability, recovery, atomicity, replication, and CDC. Understand it and you understand how databases keep the promise that "committed" means "permanent." We cover durability, replication, and the guarantees behind them across System Design Fundamentals.
Go deeper
- Write-ahead logging (Wikipedia) - the core durability technique.
- PostgreSQL - Write-Ahead Logging - a real database's WAL, documented.
- ACID (Wikipedia) - durability among the transaction guarantees.
- Designing Data-Intensive Applications - logs, recovery, and replication in depth.