Saga pattern for distributed transactions in Kafka - Step-by-Step Execution

Process Flow - Saga pattern for distributed transactions

Order Service publishes event

↓

Kafka: saga-events topic

↓

Payment Service consumes

↓

Publish [Publish compensation

↓

Kafka: saga-compensation

↓

Order Service rolls back

↓

Transaction complete

The order service starts a saga by publishing an event to Kafka. Each downstream service consumes, processes, and publishes result events. On failure, compensation events trigger rollback of previously completed steps.

Execution Sample

Kafka

# Step 1: Create saga topics
kafka-topics.sh --create --topic saga-events --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
kafka-topics.sh --create --topic saga-compensation --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1

# Step 2: Publish order-created event
echo '{"order_id":"ORD-001","step":"order-created","status":"success"}' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic saga-events

# Step 3: Consume and verify
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic saga-events --from-beginning --max-messages 1

# Step 4: Simulate payment failure - publish compensation
echo '{"order_id":"ORD-001","step":"payment-failed","action":"rollback-order"}' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic saga-compensation

# Step 5: Verify compensation event
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic saga-compensation --from-beginning --max-messages 1

This demonstrates a simplified saga flow: creating coordination topics, publishing an order event, simulating a payment failure, and publishing a compensation event for rollback.

Process Table

Step	Action	saga-events Topic	saga-compensation Topic	Transaction State	Next Step
1	Create saga-events and saga-compensation topics	Created (empty)	Created (empty)	Not started	Publish order event
2	Order Service publishes order-created event	1 message (ORD-001, order-created)	Empty	Started	Payment Service consumes
3	Payment Service consumes order event	1 message (consumed)	Empty	Processing payment	Process payment
4	Payment fails — publish compensation event	1 message	1 message (ORD-001, rollback-order)	Compensating	Order Service consumes compensation
5	Order Service consumes compensation event	1 message	1 message (consumed)	Rolled back	Transaction ends
6	Transaction complete — order cancelled	1 message	1 message	Failed (compensated)	End

💡 Saga completed with compensation. Order rolled back after payment failure. Both topics retain event history.

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	Final
orderStatus	None	Created	Created	Compensating	Cancelled
paymentStatus	None	None	Processing	Failed	Failed
sagaEventsCount	0	1	1	1	1
compensationCount	0	0	0	1	1
transactionState	Not started	Started	Processing	Compensating	Rolled back

Key Moments - 3 Insights

Why does the saga use two separate Kafka topics instead of one?

Separating saga-events (forward flow) from saga-compensation (rollback flow) prevents consumers from mixing up progress events with rollback commands. Each topic has its own consumer group, so services only listen to the events relevant to their role.

What happens if the compensation event itself fails to publish?

Kafka's producer retries and acknowledgment (acks=all) ensure the compensation event is durably written. If the producer crashes, the saga orchestrator can detect the incomplete saga via the transaction store and re-publish the compensation event — this is why idempotent consumers are critical.

How is this different from a two-phase commit?

Two-phase commit locks resources across all services until all agree to commit — creating tight coupling and blocking. The saga pattern uses eventual consistency: each service commits locally and compensates on failure. Kafka decouples the services so they never block each other.

Visual Quiz - 3 Questions

Look at the execution table. At which step does the transaction state change to 'Compensating'?

A Step 2 — when the order event is published

B Step 3 — when payment service consumes

C Step 4 — when compensation event is published

D Step 6 — when transaction completes

💡 Hint

Check the Transaction State column — it changes to Compensating when the rollback event enters the compensation topic.

From the variable tracker, what is the compensationCount after the payment failure at Step 4?

A 0

B 1

C 2

D 3

💡 Hint

Check the compensationCount row — it increments from 0 to 1 when the compensation event is published.

If Payment Service succeeded but Inventory Service failed, what would change in the flow?

A Two compensation events — one for payment rollback and one for order rollback

B Only the order gets rolled back

C No compensation needed since payment succeeded

D The entire Kafka cluster restarts

💡 Hint

Each completed saga step needs its own compensation. If inventory fails after payment succeeded, both payment and order must be compensated in reverse order.

Concept Snapshot

The saga pattern coordinates distributed transactions without distributed locks.
Each microservice commits locally and publishes events to Kafka.
On failure, compensation events trigger rollback of completed steps.
Use separate topics for forward events and compensation events.
Idempotent consumers handle retries safely.
Kafka guarantees durable delivery of saga coordination messages.

Full Transcript

This visual execution shows how the saga pattern works with Kafka for distributed transactions. The order service starts the saga by publishing an order-created event to the saga-events topic. The payment service consumes this event and attempts to process payment. When payment fails, instead of rolling back everything in a single transaction, the payment service publishes a compensation event to the saga-compensation topic. The order service consumes this compensation event and rolls back the order. Each step is tracked through the variable tracker showing how order status, payment status, and transaction state evolve. The key insight is that saga uses eventual consistency through events rather than distributed locks, making it scalable and resilient.

Practice

1. What is the main purpose of the Saga pattern in distributed systems?

easy

A. To encrypt messages between microservices

B. To speed up database queries by caching results

C. To break a large transaction into smaller steps with compensations

D. To create backups of all data in real-time

Solution

1. Step 1: Understand the Saga pattern concept

   The Saga pattern divides a big transaction into smaller, manageable steps that can be undone if needed.

2. Step 2: Identify the main goal

   Its goal is to maintain data consistency across services without locking resources for a long time.

3. Final Answer:

   To break a large transaction into smaller steps with compensations -> Option C

4. Quick Check:

   Saga pattern = smaller steps with undo [OK]

Hint: Saga splits big tasks into small reversible steps [OK]

Common Mistakes:

• Confusing Saga with caching or encryption
• Thinking Saga creates backups
• Assuming Saga speeds up queries

2. Which Kafka feature is most important for implementing the Saga pattern?

easy

A. Kafka Streams for data encryption

B. Kafka Consumer groups for load balancing

C. Kafka Connect for database backups

D. Kafka topics for event communication

Solution

1. Step 1: Identify Kafka components used in Saga

   Saga relies on events to coordinate steps, so Kafka topics are used to send these events.

2. Step 2: Exclude unrelated Kafka features

   Encryption, backups, and load balancing are not the main focus for Saga coordination.

3. Final Answer:

   Kafka topics for event communication -> Option D

4. Quick Check:

   Kafka topics = event communication [OK]

Hint: Saga uses Kafka topics to send events between services [OK]

Common Mistakes:

• Confusing Kafka Streams with event messaging
• Thinking Kafka Connect handles Saga coordination
• Mixing consumer groups with transaction steps

3. Given a Saga with three steps: A, B, and C, where each step publishes an event to Kafka, what happens if step B fails after step A succeeds?

medium

A. Step A is compensated by sending a rollback event

B. Step C executes anyway to complete the transaction

C. The entire Saga is aborted without compensation

D. Step B retries indefinitely without affecting others

Solution

1. Step 1: Understand failure handling in Saga

   If a step fails, previous successful steps must be undone using compensating actions.

2. Step 2: Apply to the scenario

   Since step B failed after A succeeded, step A must be compensated by sending a rollback event.

3. Final Answer:

   Step A is compensated by sending a rollback event -> Option A

4. Quick Check:

   Failure in step B triggers compensation for step A [OK]

Hint: Failed step triggers compensation for prior steps [OK]

Common Mistakes:

• Assuming next steps run despite failure
• Thinking Saga aborts without compensation
• Believing retries never stop

4. A developer implemented a Saga using Kafka but notices that compensating events are not triggered on failure. What is the most likely cause?

medium

A. The service does not listen to failure events to trigger compensation

B. Kafka topics are not partitioned correctly

C. Kafka consumer group ID is missing

D. Kafka brokers are overloaded

Solution

1. Step 1: Identify compensation trigger mechanism

   Compensation happens when a service listens for failure events and reacts by sending rollback events.

2. Step 2: Analyze the problem

   If compensations are missing, likely the service is not listening or handling failure events properly.

3. Final Answer:

   The service does not listen to failure events to trigger compensation -> Option A

4. Quick Check:

   Missing compensation = no failure event listener [OK]

Hint: Compensation needs failure event listeners active [OK]

Common Mistakes:

• Blaming Kafka partitions for compensation logic
• Confusing consumer group ID with compensation triggers
• Assuming broker overload causes missing compensations

5. You design a Saga with multiple microservices communicating via Kafka. To ensure scalability and fault tolerance, which approach is best?

hard

A. Store Saga state only in a centralized database without event logs

B. Use separate Kafka topics per Saga step and implement idempotent consumers

C. Use a single Kafka topic for all Saga events without consumer groups

D. Send synchronous HTTP calls between services instead of Kafka events

Solution

1. Step 1: Consider scalability and fault tolerance needs

   Separate topics per step allow independent scaling and isolation of events.

2. Step 2: Ensure idempotency in consumers

   Idempotent consumers prevent duplicate processing on retries, improving fault tolerance.

3. Step 3: Evaluate other options

   Single topic without consumer groups limits scalability; centralized DB without events loses Saga benefits; synchronous calls reduce fault tolerance.

4. Final Answer:

   Use separate Kafka topics per Saga step and implement idempotent consumers -> Option B

5. Quick Check:

   Separate topics + idempotent consumers = scalable Saga [OK]

Hint: Separate topics and idempotent consumers scale Saga well [OK]

Common Mistakes:

• Using one topic for all events limits scaling
• Ignoring idempotency causes duplicate side effects
• Relying on synchronous calls reduces fault tolerance