System Design
The Saga Pattern: Distributed Transactions Without Locks or Rollback
Large systems don’t panic when things fail — they’re built to expect it. The Saga pattern is how they run transactions across many services with no global lock, no full rollback, and nothing ever blocked. Here’s how it actually works.
Here’s the secret behind systems like Amazon that most people never realize: they don’t panic when things fail. They’re built from the ground up with the expectation that failures will happen - a service will time out, a payment will fail, a network call will drop. Instead of trying to prevent every possible failure (impossible at scale), these systems are designed to handle failure gracefully - detect it, compensate for it, and keep moving forward. This resilience-first mindset is what separates large-scale distributed systems from traditional monoliths, and the design pattern that makes it work is the Saga pattern. This post is a deep dive into how it runs transactions across many services with no global lock, no full rollback, and nothing ever blocked. (This builds on the previous post about eventual consistency, which introduced the saga as the mechanism behind it - here we go deep on the pattern itself.)
Recap: from eventual consistency to the pattern that powers it
Quickly, where we left off. We learned that distributed services don’t stay consistent immediately - they become eventually consistent through compensation mechanisms. When a payment succeeds but an order is marked failed, the system doesn’t stay broken; it detects the mismatch and compensates (refund, reconcile) until all services converge. That’s the what and the why. Today is the how - the design pattern that orchestrates this entire process: the Saga pattern.
What is the Saga pattern?
A Saga is a design pattern used in microservices architecture to manage long-running distributed transactions - transactions that span multiple independent services. The core idea addresses a problem we’ve seen: you can’t wrap everything in one large atomic transaction across multiple independent services (impractical at scale, since each service has its own database and runs independently). So instead, a Saga breaks the transaction into a sequence of smaller, independent steps. Each step is a local transaction within its own service, and - this is key - each step commits on its own. So instead of one giant all-or-nothing transaction, a Saga is a chain of local transactions: the order service does its local transaction and commits, the inventory service does its local transaction and commits, the payment service does its local transaction and commits. Each is a normal, ACID-compliant transaction within its own database - the Saga just sequences them across services. The “distributed transaction” becomes a series of local ones, strung together.
No global lock, no full rollback
This is where the Saga fundamentally departs from the traditional model, and understanding the contrast is understanding the pattern. In a traditional monolithic transaction, if one step fails, the entire transaction is rolled back - undone atomically - often using a global lock that holds everything in place until the whole transaction either commits or aborts as a unit. It’s clean and simple, but it requires locking and coordinating everything together. The Saga works differently, in two crucial ways:
- There is no global lock across services. No single lock holds all the services in place waiting for a distributed commit. Each service commits independently, so the system is never fully locked waiting for a coordinated all-or-nothing decision across services.
- There is no single rollback of everything. When something fails, the Saga doesn’t magically rewind all the committed local transactions (it can’t - they already committed independently, in separate databases). Instead, it does something different: compensation. The absence of a global lock is a big deal. It’s precisely what lets each service stay independent, available, and fast - no service is frozen waiting for a distributed commit that might never come. The trade-off is that you give up the neat atomic rollback, and in its place you need a mechanism to handle failures forward. That mechanism is compensation.
Compensating steps: how Saga handles failure
When a step in the Saga fails, instead of rolling back all previous steps like a traditional transaction, the Saga triggers compensating actions - specific operations designed to undo or neutralize the effect of previously completed steps. Here’s the subtle but profound distinction: compensation moves the system forward toward a consistent state, rather than rewinding it. A rollback pretends the work never happened (rewinds time). A compensation acknowledges the work did happen and performs a new action to counteract it. If a payment was charged and then a later step fails, you don’t “un-charge” the payment (you can’t reach back into an already-committed transaction in another service) - you issue a refund, which is a new, forward action that neutralizes the charge. The system progresses to consistency through corrective actions, not by reversing time. Each completed step in a Saga has a matching compensating action defined for it - a “how to undo this” counterpart. If the Saga gets partway through and a step fails, it runs the compensating actions for the steps that already completed, typically in reverse order, unwinding the work through forward corrections.
graph TD
S1[Create Order ✓] --> S2[Reserve Inventory ✓]
S2 --> S3[Charge Payment]
S3 -->|FAILS| F[Failure detected]
F --> C1[Compensate: Release Inventory]
C1 --> C2[Compensate: Cancel Order]
C2 --> DONE[System consistent again]
S3 -->|Succeeds| OK[Order placed]
This flowchart shows a Saga with compensation. The steps run forward: create the order, reserve inventory, then charge the payment. If charging the payment fails, the failure is detected, and the Saga runs compensating actions for the completed steps in reverse: release the inventory, then cancel the order, bringing the system back to a consistent state. If the payment succeeds instead, the order is placed. Compensation moves the system forward to consistency through corrective actions, rather than rolling back.
Real-world examples of compensation
Two concrete examples make this click:
- Payment fails after the order is created → the compensating step is to cancel the order. The order was already committed (it exists in the order service’s database), so you can’t un-create it atomically - you perform a new action, cancelling it, to neutralize its effect.
- Inventory update fails after payment has been processed → the compensating step is to refund the payment. The payment already went through, so you don’t reverse that transaction - you issue a refund, a forward action that counteracts the charge. Notice the pattern: each failure triggers a targeted, specific correction - cancel this order, refund this payment - not a blanket undo of everything. Each compensating action is designed for the specific step it neutralizes. This is more work to design than a blanket rollback (you have to define a compensating action for each step), but it’s what makes distributed transactions possible without a global lock or a distributed rollback.
The key principle: nothing is blocked
Here’s the defining characteristic of the Saga pattern, the property that everything else serves: no service is ever blocked waiting for another. Each step executes independently, commits locally, and moves on. If something goes wrong downstream, compensation kicks in - but crucially, while all this happens, no service is frozen holding a lock and waiting. The order service commits and is free; it doesn’t sit locked waiting to hear whether the payment succeeds. The system keeps moving. It doesn’t freeze, and - critically - it can’t deadlock, because deadlocks require threads/services to hold resources while waiting on each other, and in a Saga, nobody holds a global lock while waiting. This is the resilience-first mindset made concrete. A traditional distributed transaction with a global lock is fragile: if any participant is slow or down, everyone waits, and the whole thing can stall or deadlock. A Saga has no such choke point - every service acts independently and the system stays responsive even as failures happen and compensations run. “Nothing is blocked” is why Sagas scale and stay available where distributed locking would grind to a halt.
Important: The Saga pattern’s genius is reframing “handle failure” from rewind everything to keep moving and correct forward. Traditional transactions handle failure by rolling back under a global lock - safe, but it requires locking and coordinating everything, which doesn’t scale across services. Sagas handle failure by compensating - each service commits independently (nothing blocked), and on failure, targeted compensating actions neutralize the completed steps. You trade atomic rollback for forward compensation, and in return you get independence, availability, and no distributed deadlocks. That trade is the heart of resilient distributed design.
Saga vs. traditional ACID transactions
It’s worth placing the Saga against the ACID transactions it replaces across services. Traditional ACID transactions guarantee Atomicity (all or nothing), Consistency, Isolation, and Durability - but they only work well within a single database. ACID is the gold standard inside one database; it’s exactly what you can’t cheaply extend across many independent services. The Saga pattern trades strict atomicity for availability and partition tolerance. Instead of atomic all-or-nothing with rollback, it relies on eventual consistency and compensation. You give up the guarantee that everything is instantly consistent and atomically reversible; you gain the ability for services to stay independent, available, and scalable. This is the CAP theorem trade-off that distributed systems must make: at scale, when networks can partition, you can’t have perfect consistency and high availability, so the Saga chooses availability (services keep working independently) and accepts eventual consistency (they converge via compensation). (See the CAP theorem and ACID transactions guides.)
| ACID transaction | Saga pattern | |
|---|---|---|
| Scope | Single database | Across many services |
| Atomicity | Atomic (all or nothing) | Per-step local commits |
| On failure | Roll back everything (global lock) | Compensate completed steps (no lock) |
| Consistency | Immediate | Eventual |
| Blocking | Holds locks during transaction | Nothing blocked |
| Trades | Consistency, at the cost of scale | Availability & scale, at the cost of immediate consistency |
Two flavors of Saga: choreography vs. orchestration
Sagas come in two implementation styles, and the choice matters more than it first appears: Choreography-based Sagas. There’s no central coordinator. Each service listens for events and decides independently what to do next. The order service publishes “order created”; the inventory service hears it, does its step, publishes “inventory reserved”; the payment service hears that and proceeds. The coordination logic is distributed across the services, connected by events (flowing through a message broker - see the message queues material). Choreography is simpler to build initially and loosely coupled - but the overall flow is harder to track, because no single place describes the whole transaction; you have to piece it together from events scattered across services. Orchestration-based Sagas. A central orchestrator service tells each participant what step to execute, waits for the result, decides the next step, and handles the compensation logic if something fails. The whole flow lives in one place - the orchestrator. Orchestration is more complex to build (you have to create and maintain the orchestrator), but it’s far easier to monitor, debug, and maintain, because the entire transaction’s logic and state are centralized and visible.
graph TD
subgraph CH["Choreography: no coordinator"]
A1[Order] -->|event| A2[Inventory] -->|event| A3[Payment]
end
subgraph OR["Orchestration: central coordinator"]
B0[Orchestrator]
B0 -->|do step| B1[Order]
B0 -->|do step| B2[Inventory]
B0 -->|do step| B3[Payment]
B1 -.->|result| B0
B2 -.->|result| B0
B3 -.->|result| B0
end
This diagram shows the two Saga styles. In choreography, there's no coordinator: the order service emits an event that the inventory service reacts to, which emits an event the payment service reacts to - coordination is a distributed chain of events. In orchestration, a central orchestrator directs each participant, telling the order, inventory, and payment services what step to execute and receiving their results, keeping the whole flow and its compensation logic in one place.
Tip: The practical guidance: most production systems at scale prefer orchestration for critical flows. Here’s why. Choreography’s simplicity is appealing for small or simple Sagas, but as a flow grows, the “where is this transaction right now, and why did it fail?” question becomes very hard to answer when the logic is scattered across many services reacting to events. For critical flows (like payments), you need to be able to monitor, debug, and reason about the whole transaction - and orchestration gives you that, because one orchestrator owns the flow and its state. The extra complexity of building an orchestrator pays for itself in observability and maintainability the moment something goes wrong at 3 a.m. Choreography for simple, loosely-coupled flows; orchestration for critical, complex ones.
# Orchestration-style Saga logic (conceptual)
completed = []
try:
createOrder(); completed.append(cancelOrder) # step + its compensation
reserveInventory(); completed.append(releaseInventory)
chargePayment(); completed.append(refundPayment)
placeOrder() # all steps succeeded
except StepFailed:
for compensate in reversed(completed): # undo completed steps, in reverse
compensate()
Interview: The Saga pattern is high-value system design material. A strong answer: a Saga manages a distributed transaction across microservices as a sequence of local transactions, each committing independently - no global lock, no distributed rollback. On failure, it runs compensating actions (forward corrections like “cancel order” or “refund payment”) to neutralize completed steps, rather than rolling back. Its defining property is that nothing is blocked - services act independently, so the system stays available and can’t deadlock. Explain the choreography vs. orchestration trade-off (events-and-no-coordinator vs. central-coordinator; simpler vs. more observable) and note that critical flows usually favor orchestration. Tie it to ACID and CAP (trading atomicity for availability), and you’ve shown genuine distributed-systems maturity.
The bottom line
The Saga pattern is how distributed systems run transactions across many independent services without the single atomic transaction they can’t have. It breaks the transaction into a sequence of local transactions, each committing independently in its own service - so there’s no global lock holding everything, and no single rollback of everything. When a step fails, the Saga triggers compensating actions - targeted forward corrections like cancelling an order or refunding a payment - that neutralize completed steps and move the system toward consistency rather than rewinding it. Its defining property is that nothing is ever blocked: services act independently, stay available, and can’t deadlock. This trades ACID’s strict atomicity for availability and scale (the CAP trade-off), relying on eventual consistency and compensation instead of rollback. And it comes in two flavors - choreography (event-driven, no coordinator, simpler but harder to track) and orchestration (central coordinator, more complex but far more observable, and preferred for critical flows). Master the Saga, and you understand how the world’s largest systems stay correct while never standing still. We cover the saga pattern, distributed transactions, and resilient design across System Design Fundamentals.
Go deeper
- Saga pattern (microservices.io) - the canonical reference on sagas, compensation, and the two styles.
- CAP theorem (Wikipedia) - the availability-vs-consistency trade-off the Saga makes.
- ACID (Wikipedia) - the single-database guarantees the Saga replaces across services.
- Designing Data-Intensive Applications - distributed transactions and their alternatives in depth.