Saga pattern for distributed transactions in Kafka - Mini Project: Build & Apply

Implementing the Saga Pattern for Distributed Transactions with Kafka

📖 Scenario: Your e-commerce platform uses separate microservices for orders, payments, and inventory. When a customer places an order, all three services must succeed or the entire transaction must roll back. You need to coordinate this distributed transaction using the saga pattern with Kafka as the event bus.

🎯 Goal: Build a Kafka-based saga coordination system by creating the required topics, publishing an order event, simulating a saga step, and implementing a compensation event for rollback.

📋 What You'll Learn

Create Kafka topics for saga events and compensation events

Publish an order-created event to start the saga

Consume the event and simulate a payment processing step

Publish a compensation event to simulate rollback on failure

Verify the full saga flow using console tools

💡 Why This Matters

🌍 Real World

The saga pattern coordinates distributed transactions across microservices without distributed locks. Kafka provides reliable event delivery for saga steps and compensation actions.

💼 Career

Backend and DevOps engineers working with microservices architectures implement saga patterns using Kafka to maintain data consistency across services.

1

Create Kafka topics for saga coordination

Create two Kafka topics: saga-events with 3 partitions for saga step events, and saga-compensation with 3 partitions for rollback compensation events. Use replication factor 1 for local development.

Kafka

# Create saga event topics
# Your code here

Need a hint?

Use kafka-topics.sh --create twice — once for saga-events and once for saga-compensation.

2

Publish an order-created event to start the saga

Use the Kafka console producer to publish a JSON event to the saga-events topic. The event should contain an order ID, step name, and status: {"order_id":"ORD-001","step":"order-created","status":"success"}.

Kafka

kafka-topics.sh --create --topic saga-events --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
kafka-topics.sh --create --topic saga-compensation --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
# Publish order-created event here

Need a hint?

Pipe a JSON string into kafka-console-producer.sh targeting the saga-events topic.

3

Consume the event and publish a payment failure for compensation

First consume from saga-events to verify the order event arrived. Then simulate a payment failure by publishing a compensation event to saga-compensation topic: {"order_id":"ORD-001","step":"payment-failed","action":"rollback-order"}.

Kafka

kafka-topics.sh --create --topic saga-events --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
kafka-topics.sh --create --topic saga-compensation --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
echo '{"order_id":"ORD-001","step":"order-created","status":"success"}' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic saga-events
# Consume saga event and publish compensation here

Need a hint?

Use kafka-console-consumer.sh --from-beginning --max-messages 1 to read the event, then pipe a compensation JSON into the compensation topic.

4

Verify the compensation event was received

Consume from the saga-compensation topic to verify the rollback event is available for the order service to process.

Kafka

kafka-topics.sh --create --topic saga-events --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
kafka-topics.sh --create --topic saga-compensation --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
echo '{"order_id":"ORD-001","step":"order-created","status":"success"}' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic saga-events
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic saga-events --from-beginning --max-messages 1
echo '{"order_id":"ORD-001","step":"payment-failed","action":"rollback-order"}' | kafka-console-producer.sh --bootstrap-server localhost:9092 --topic saga-compensation
# Verify compensation event here

Need a hint?

Use kafka-console-consumer.sh --from-beginning --max-messages 1 on the saga-compensation topic.