Saga pattern for distributed transactions in Kafka - Step-by-Step Execution

Process Flow - Saga pattern for distributed transactions

Order Service publishes event

↓

Kafka: saga-events topic

↓

Payment Service consumes

↓

Publish [Publish compensation

↓

Kafka: saga-compensation

↓

Order Service rolls back

↓

Transaction complete

The order service starts a saga by publishing an event to Kafka. Each downstream service consumes, processes, and publishes result events. On failure, compensation events trigger rollback of previously completed steps.

Execution Sample

Kafka

# Step 1: Create saga topics
kafka-topics.sh --create --topic saga-events --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
kafka-topics.sh --create --topic saga-compensation --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1

# Step 2: Publish order-created event
echo '{"order_id":"ORD-001","step":"order-created","status":"success"}' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic saga-events

# Step 3: Consume and verify
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic saga-events --from-beginning --max-messages 1

# Step 4: Simulate payment failure - publish compensation
echo '{"order_id":"ORD-001","step":"payment-failed","action":"rollback-order"}' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic saga-compensation

# Step 5: Verify compensation event
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic saga-compensation --from-beginning --max-messages 1

This demonstrates a simplified saga flow: creating coordination topics, publishing an order event, simulating a payment failure, and publishing a compensation event for rollback.

Process Table

Step	Action	saga-events Topic	saga-compensation Topic	Transaction State	Next Step
1	Create saga-events and saga-compensation topics	Created (empty)	Created (empty)	Not started	Publish order event
2	Order Service publishes order-created event	1 message (ORD-001, order-created)	Empty	Started	Payment Service consumes
3	Payment Service consumes order event	1 message (consumed)	Empty	Processing payment	Process payment
4	Payment fails — publish compensation event	1 message	1 message (ORD-001, rollback-order)	Compensating	Order Service consumes compensation
5	Order Service consumes compensation event	1 message	1 message (consumed)	Rolled back	Transaction ends
6	Transaction complete — order cancelled	1 message	1 message	Failed (compensated)	End

💡 Saga completed with compensation. Order rolled back after payment failure. Both topics retain event history.

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	Final
orderStatus	None	Created	Created	Compensating	Cancelled
paymentStatus	None	None	Processing	Failed	Failed
sagaEventsCount	0	1	1	1	1
compensationCount	0	0	0	1	1
transactionState	Not started	Started	Processing	Compensating	Rolled back

Key Moments - 3 Insights

Why does the saga use two separate Kafka topics instead of one?

Separating saga-events (forward flow) from saga-compensation (rollback flow) prevents consumers from mixing up progress events with rollback commands. Each topic has its own consumer group, so services only listen to the events relevant to their role.

What happens if the compensation event itself fails to publish?

Kafka's producer retries and acknowledgment (acks=all) ensure the compensation event is durably written. If the producer crashes, the saga orchestrator can detect the incomplete saga via the transaction store and re-publish the compensation event — this is why idempotent consumers are critical.

How is this different from a two-phase commit?

Two-phase commit locks resources across all services until all agree to commit — creating tight coupling and blocking. The saga pattern uses eventual consistency: each service commits locally and compensates on failure. Kafka decouples the services so they never block each other.

Visual Quiz - 3 Questions

Look at the execution table. At which step does the transaction state change to 'Compensating'?

A Step 2 — when the order event is published

B Step 3 — when payment service consumes

C Step 4 — when compensation event is published

D Step 6 — when transaction completes

💡 Hint

Check the Transaction State column — it changes to Compensating when the rollback event enters the compensation topic.

From the variable tracker, what is the compensationCount after the payment failure at Step 4?

A 0

B 1

C 2

D 3

💡 Hint

Check the compensationCount row — it increments from 0 to 1 when the compensation event is published.

If Payment Service succeeded but Inventory Service failed, what would change in the flow?

A Two compensation events — one for payment rollback and one for order rollback

B Only the order gets rolled back

C No compensation needed since payment succeeded

D The entire Kafka cluster restarts

💡 Hint

Each completed saga step needs its own compensation. If inventory fails after payment succeeded, both payment and order must be compensated in reverse order.

Concept Snapshot

The saga pattern coordinates distributed transactions without distributed locks.
Each microservice commits locally and publishes events to Kafka.
On failure, compensation events trigger rollback of completed steps.
Use separate topics for forward events and compensation events.
Idempotent consumers handle retries safely.
Kafka guarantees durable delivery of saga coordination messages.

Full Transcript

This visual execution shows how the saga pattern works with Kafka for distributed transactions. The order service starts the saga by publishing an order-created event to the saga-events topic. The payment service consumes this event and attempts to process payment. When payment fails, instead of rolling back everything in a single transaction, the payment service publishes a compensation event to the saga-compensation topic. The order service consumes this compensation event and rolls back the order. Each step is tracked through the variable tracker showing how order status, payment status, and transaction state evolve. The key insight is that saga uses eventual consistency through events rather than distributed locks, making it scalable and resilient.