Event choreography vs orchestration in Kafka - Hands-On Comparison

Event Choreography vs Orchestration with Kafka

📖 Scenario: Imagine you are building a simple online store system where different services communicate using Kafka events. You want to understand the difference between event choreography and orchestration by implementing both approaches.

🎯 Goal: Build two simple Kafka event-driven flows: one using event choreography where services react to events independently, and one using orchestration where a central controller manages the flow.

📋 What You'll Learn

Create Kafka topics for order events

Set up event producers and consumers

Implement event choreography with independent consumers

Implement event orchestration with a central orchestrator service

Print messages showing the flow of events

💡 Why This Matters

🌍 Real World

Event-driven systems like online stores, payment processing, and microservices often use Kafka to communicate asynchronously.

💼 Career

Understanding event choreography and orchestration helps you design scalable, maintainable distributed systems and work effectively with Kafka in real jobs.

1

Create Kafka topics and initial order event

Create a Kafka topic called orders and produce an initial order event with the key order-123 and value {"status": "created"}.

Kafka

# Your code here to create topic 'orders' and produce initial order event

Hint

Use KafkaProducer to send a JSON event with key order-123 to the orders topic.

2

Set up a configuration variable for event status to listen for

Create a variable called listen_status and set it to created to filter events with this status.

Kafka

from kafka import KafkaProducer
import json

producer = KafkaProducer(bootstrap_servers='localhost:9092', value_serializer=lambda v: json.dumps(v).encode('utf-8'))

# Produce initial order created event
producer.send('orders', key=b'order-123', value={'status': 'created'})
producer.flush()

# Create a variable called listen_status and set it to 'created'
# Your code here

Hint

Just create a variable named listen_status and assign the string 'created' to it.

3

Implement event choreography consumer reacting to 'created' events

Write a Kafka consumer that listens to the orders topic and prints Order {key} received with status {status} only if the event's status matches the listen_status variable.

Kafka

from kafka import KafkaProducer
import json
from kafka import KafkaConsumer

producer = KafkaProducer(bootstrap_servers='localhost:9092', value_serializer=lambda v: json.dumps(v).encode('utf-8'))

# Produce initial order created event
producer.send('orders', key=b'order-123', value={'status': 'created'})
producer.flush()

# Configuration variable to listen for specific event status
listen_status = 'created'

# Create a KafkaConsumer to listen to 'orders' topic
# For each message, decode JSON and check if status equals listen_status
# If yes, print the message as specified
# Your code here

Hint

Use KafkaConsumer to read from orders. Decode the key and value, check if status matches listen_status, then print the message.

4

Print the final message showing event choreography output

Print the exact message Order order-123 received with status created to show the event choreography flow.

Kafka

from kafka import KafkaProducer
import json
from kafka import KafkaConsumer

producer = KafkaProducer(bootstrap_servers='localhost:9092', value_serializer=lambda v: json.dumps(v).encode('utf-8'))

# Produce initial order created event
producer.send('orders', key=b'order-123', value={'status': 'created'})
producer.flush()

# Configuration variable to listen for specific event status
listen_status = 'created'

# Event choreography consumer
consumer = KafkaConsumer('orders', bootstrap_servers='localhost:9092', auto_offset_reset='earliest', value_deserializer=lambda v: json.loads(v))

for message in consumer:
    key = message.key.decode('utf-8') if message.key else 'unknown'
    status = message.value.get('status', '')
    if status == listen_status:
        print(f"Order {key} received with status {status}")
        break

# Print the final message below
# Your code here

Hint

Just print the exact message to show the event choreography result.

Practice

1. What is the main difference between event choreography and event orchestration in Kafka-based systems?

easy

A. Choreography lets services react independently; orchestration uses a central controller.

B. Choreography requires a central controller; orchestration lets services react independently.

C. Both use a central controller but differ in message format.

D. Choreography is synchronous; orchestration is asynchronous.

Solution

1. Step 1: Understand event choreography

   Event choreography means each service listens and reacts to events on its own without a central manager.

2. Step 2: Understand event orchestration

   Event orchestration means a central controller tells services what to do and when, managing the flow.

3. Final Answer:

   Choreography lets services react independently; orchestration uses a central controller. -> Option A

4. Quick Check:

   Choreography = independent reaction, Orchestration = central control [OK]

Hint: Choreography = no boss; orchestration = boss controls flow [OK]

Common Mistakes:

• Confusing which uses a central controller
• Thinking both require central control
• Mixing synchronous and asynchronous concepts

2. Which Kafka component is typically used in event orchestration to coordinate services?

easy

A. Kafka Topics with independent consumers

B. A central orchestrator service producing commands

C. Kafka Connect for data integration

D. Kafka Streams for independent event processing

Solution

1. Step 1: Identify orchestration role

   Orchestration needs a central service to control and send commands to other services.

2. Step 2: Match Kafka components

   Kafka Topics and Streams are used for event flow, but orchestration requires a central orchestrator producing commands.

3. Final Answer:

   A central orchestrator service producing commands -> Option B

4. Quick Check:

   Orchestration = central orchestrator service [OK]

Hint: Orchestration needs a central command sender [OK]

Common Mistakes:

• Choosing Kafka Topics as the controller
• Confusing Kafka Connect with orchestration
• Thinking Kafka Streams alone orchestrates

3. Given this Kafka event flow in choreography style:

Service A publishes event 'OrderCreated' to topic 'orders'.
Service B listens to 'orders' and publishes 'PaymentProcessed' to topic 'payments'.
Service C listens to 'payments' and updates inventory.

What happens if Service B fails to publish 'PaymentProcessed' event?

medium

A. Service C will not update inventory because it never receives the event.

B. Service C updates inventory anyway because it listens to 'orders'.

C. Service A retries publishing 'OrderCreated' event automatically.

D. A central orchestrator will resend 'PaymentProcessed' event.

Solution

1. Step 1: Trace event flow in choreography

   Service A publishes 'OrderCreated', Service B reacts and publishes 'PaymentProcessed', Service C listens to 'payments'.

2. Step 2: Analyze failure impact

   If Service B fails to publish 'PaymentProcessed', Service C never receives that event and cannot update inventory.

3. Final Answer:

   Service C will not update inventory because it never receives the event. -> Option A

4. Quick Check:

   Missing event means downstream service does not act [OK]

Hint: In choreography, missing event stops downstream actions [OK]

Common Mistakes:

• Assuming Service C listens to 'orders' directly
• Thinking Service A retries automatically
• Expecting a central orchestrator in choreography

4. You have this orchestration code snippet controlling Kafka services:

orchestrator.sendCommand('ServiceB', 'ProcessPayment')
orchestrator.sendCommand('ServiceC', 'UpdateInventory')

But Service C updates inventory before Service B processes payment. What is the likely problem?

medium

A. Service C is ignoring commands from the orchestrator.

B. Service B is not subscribed to the correct Kafka topic.

C. Commands are sent asynchronously without waiting for Service B to finish.

D. Kafka topics are misconfigured causing message loss.

Solution

1. Step 1: Understand orchestration command flow

   Orchestrator sends commands to services in order, expecting sequential execution.

2. Step 2: Identify asynchronous issue

   If commands are sent asynchronously without waiting for Service B to complete, Service C may act too early.

3. Final Answer:

   Commands are sent asynchronously without waiting for Service B to finish. -> Option C

4. Quick Check:

   Orchestration needs ordered, awaited commands [OK]

Hint: Orchestration requires waiting for each command to finish [OK]

Common Mistakes:

• Blaming Kafka topic subscription
• Assuming Service C ignores commands
• Thinking message loss causes order issue

5. You want to design a Kafka system for an online store where payment processing must strictly happen before inventory update, but you want to avoid a central orchestrator. Which approach fits best and how to ensure order?

hard

A. Use choreography but rely on manual retries to fix order issues.

B. Use event orchestration with a central controller sending commands in order to services.

C. Use orchestration but allow services to process events independently without waiting.

D. Use event choreography with services emitting events and consumers processing in sequence using Kafka partitions and consumer groups.

Solution

1. Step 1: Identify requirement

   Payment must happen before inventory update, but no central orchestrator allowed.

2. Step 2: Choose choreography with ordering

   Event choreography fits because services react independently. To ensure order, use Kafka partitions and consumer groups to process events in sequence.

3. Final Answer:

   Use event choreography with services emitting events and consumers processing in sequence using Kafka partitions and consumer groups. -> Option D

4. Quick Check:

   Choreography + Kafka ordering = correct sequence without central control [OK]

Hint: Choreography + Kafka partitions ensure order without central control [OK]

Common Mistakes:

• Choosing orchestration despite no central controller allowed
• Ignoring Kafka partition ordering
• Relying on manual retries for order