Distributed Transactions and Sagas: Coordinating Work Across Services

A business action can cross several systems.

Booking a trip may reserve a flight, reserve a hotel, charge a payment method, and issue confirmation. Each system owns its own data and can succeed or fail independently.

Cross-service consistency is often a managed workflow of local transactions, not one instant all-or-nothing database commit.

A saga makes progress through steps and uses compensating actions when later work fails.

A concrete example: booking a trip

A booking workflow:

1. reserves a flight,
2. reserves a hotel,
3. charges the traveler,
4. confirms the itinerary.

The flight succeeds, but the hotel is unavailable. The workflow cannot erase time and pretend the flight reservation never existed. It sends a cancellation request and tracks whether cancellation succeeds.

The final acceptable state is "no trip booked and flight released," reached through additional business actions.

Local database transactions

Inside one database, a transaction can provide atomicity:

• all included changes commit,
• or none commit.

The database controls locks, logs, and recovery for its own state.

Once work spans independent databases and providers, no single local transaction automatically controls all participants.

Why not share one database?

One shared transaction can simplify atomic changes, and splitting data prematurely may create unnecessary complexity.

Independent services may still need separate ownership because of:

• scaling,
• security boundaries,
• distinct availability needs,
• organizational responsibility,
• or external providers.

The architecture should justify distribution before paying its consistency cost.

Distributed atomic commit

Protocols such as two-phase commit attempt to coordinate one transaction across participants.

Broadly:

1. a coordinator asks each participant to prepare,
2. participants promise they can commit and hold required state,
3. the coordinator decides commit or abort,
4. participants apply that decision.

This can provide strong atomicity within supported systems.

Costs of two-phase commit

Distributed commit can:

• hold locks while waiting,
• reduce availability when a coordinator or participant is unreachable,
• increase latency,
• require compatible transaction support,
• and complicate recovery.

Many web services and external APIs do not participate in one shared transaction manager.

It remains useful in some controlled environments, but it is not a universal solution.

What is a saga?

A saga is a sequence of local transactions representing one larger business workflow.

Each step commits independently. If a later step fails, the saga executes a compensating action for earlier work where appropriate.

The system can therefore expose intermediate states such as reserving, compensating, or requiring manual review.

Compensation is not rollback

A database rollback erases uncommitted changes as if they never became visible.

A compensation is a new business action:

• cancel reservation,
• issue refund,
• release inventory,
• revoke credit,
• or mark an order void.

History remains. Fees, emails, or external observations may not be reversible.

Designing compensating actions

For every forward step, ask:

• Can it be undone?
• What does "undo" mean in the business?
• Is compensation guaranteed?
• Does it have a deadline or fee?
• Is it idempotent?
• What if compensation also fails?

Some actions should be delayed until commitment is likely because compensation is expensive or impossible.

Ordering for reversibility

Step order can reduce risk.

For example:

1. place temporary holds,
2. verify all resources,
3. commit payment,
4. finalize reservations.

Temporary, expiring holds are easier to release than fully issued nonrefundable tickets.

Workflow design should prefer reversible actions before irreversible ones.

Orchestration

In an orchestrated saga, one coordinator decides which command comes next.

It:

• stores workflow state,
• sends commands,
• receives results,
• schedules retries,
• initiates compensation,
• and exposes status.

The central view makes complex flows understandable but creates an important service that needs high reliability.

Choreography

In a choreographed saga, participants react to events.

For example:

• OrderPlaced triggers inventory reservation,
• InventoryReserved triggers payment,
• PaymentCaptured triggers fulfillment.

This removes one central coordinator but can make the overall workflow difficult to see, change, and debug as event chains grow.

Choosing orchestration or choreography

Choreography works well for simple, loosely coupled reactions.

Orchestration is often clearer when:

• steps have complex order,
• compensation matters,
• timeouts and branches exist,
• or one team owns the business process.

Hybrid designs are common. The key is preserving an understandable source of workflow truth.

Durable workflow state

A saga may run for seconds, days, or weeks.

Its state must survive process restart:

• current step,
• completed steps,
• attempt count,
• deadlines,
• identifiers,
• compensation state,
• and last error.

In-memory callbacks alone cannot safely coordinate long-lived business work.

Idempotent steps

Commands and events can be delivered more than once.

Each step should use a stable operation identifier and safely return the original result for duplicates. Compensations also need idempotency because cancellation or refund may be retried.

Duplicate protection applies to both forward and reverse paths.

Timeouts and uncertainty

A step can time out after succeeding remotely.

Before compensating, the orchestrator may need to query status. Issuing "cancel reservation" can itself be safe if cancellation is idempotent and accepts both existing and already-canceled states.

Timeout does not equal failure, so saga states must include uncertainty.

Concurrent sagas

Two workflows may compete for the same resource.

Inventory reservations, account limits, and version checks protect local invariants. A saga does not eliminate concurrency; each service must enforce its own correctness.

Optimistic concurrency can reject stale commands and trigger workflow reevaluation.

Isolation problems

Intermediate saga state can be visible.

Another workflow may observe a temporary reservation or partial account update. Techniques to reduce anomalies include:

• semantic locks,
• pending status,
• escrow or holds,
• version checks,
• and business rules that understand in-progress work.

Sagas provide atomicity through compensation, not traditional transaction isolation.

Outbox and inbox patterns

Services need reliable communication around local commits.

An outbox records outgoing messages in the same transaction as local state. An inbox deduplicates incoming messages with the local effect.

Together they reduce gaps between database state and event delivery.

Manual intervention

Some failures cannot be resolved automatically:

• provider dispute,
• expired cancellation window,
• inconsistent external status,
• or repeated compensation failure.

The workflow needs a visible manual-review state, operational tooling, audit history, and clear ownership.

Silently retrying forever is not recovery.

Observability

Track each saga by one correlation identifier.

Operators need:

• current state,
• timeline of commands and events,
• participant results,
• retries,
• deadlines,
• compensation progress,
• and manual actions.

Aggregate metrics should show stuck workflows and time spent in each stage.

Testing sagas

Test:

• every step failing,
• timeout after success,
• duplicate command and event,
• out-of-order event,
• process restart,
• compensation failure,
• concurrent workflow,
• and manual recovery.

State-machine tests can verify that every state has a valid path to completion or explicit intervention.

Knowledge check

1. Why can one local database transaction not control an external provider?
2. How does compensation differ from rollback?
3. What tradeoff separates orchestration from choreography?
4. Why do both forward and compensating steps need idempotency?
5. What isolation issue can intermediate saga state create?

The one idea to remember

A saga coordinates local transactions through durable workflow state and compensating business actions. It does not erase history or guarantee traditional isolation, so step ordering, idempotency, uncertainty, observability, and manual recovery must be designed explicitly.