Why event-driven scales applications in Kafka - See It in Action

Why Event-Driven Architecture Scales Applications

📖 Scenario: Imagine you are building a simple online store system. When a customer places an order, the system needs to process the order, update inventory, and notify the shipping department. Using event-driven architecture with Kafka helps the system handle many orders smoothly without slowing down.

🎯 Goal: Build a simple event-driven flow using Kafka concepts to understand how it helps scale applications by decoupling parts of the system.

📋 What You'll Learn

Create a Kafka topic called orders to hold order events

Create a producer that sends order messages to the orders topic

Create a consumer that listens to the orders topic and processes orders

Print the processed order details to show the flow

💡 Why This Matters

🌍 Real World

Event-driven systems like this are used in online stores, banking, and social media to handle many actions happening at once without slowing down.

💼 Career

Understanding event-driven architecture and Kafka is valuable for backend developers, system architects, and anyone working on scalable, reliable software systems.

1

Create the Kafka topic for orders

Write code to create a Kafka topic named orders that will hold order events.

Kafka

# Your code here to create the 'orders' topic

Need a hint?

Use NewTopic with name orders, 3 partitions, and replication factor 1.

2

Create a producer to send order events

Write code to create a Kafka producer that sends an order message with key order1 and value {"item": "book", "quantity": 2} to the orders topic.

Kafka

from kafka.admin import KafkaAdminClient, NewTopic

admin_client = KafkaAdminClient(bootstrap_servers="localhost:9092")

orders_topic = NewTopic(name="orders", num_partitions=3, replication_factor=1)

admin_client.create_topics(new_topics=[orders_topic], validate_only=False)

# Your code here to create a producer and send one order message

Need a hint?

Create a KafkaProducer with JSON serialization and send the order message with key order1.

3

Create a consumer to process order events

Write code to create a Kafka consumer that listens to the orders topic, reads messages, and stores them in a list called processed_orders.

Kafka

from kafka.admin import KafkaAdminClient, NewTopic
from kafka import KafkaProducer, KafkaConsumer
import json

admin_client = KafkaAdminClient(bootstrap_servers="localhost:9092")

orders_topic = NewTopic(name="orders", num_partitions=3, replication_factor=1)

admin_client.create_topics(new_topics=[orders_topic], validate_only=False)

producer = KafkaProducer(bootstrap_servers='localhost:9092', value_serializer=lambda v: json.dumps(v).encode('utf-8'), key_serializer=lambda k: k.encode('utf-8'))

order_message = {"item": "book", "quantity": 2}

producer.send('orders', key='order1', value=order_message)
producer.flush()

# Your code here to create a consumer and collect processed orders

Need a hint?

Create a KafkaConsumer subscribed to orders and append messages to processed_orders.

4

Print the processed orders

Write code to print the processed_orders list to show the processed order details.

Kafka

from kafka.admin import KafkaAdminClient, NewTopic
from kafka import KafkaProducer, KafkaConsumer
import json

admin_client = KafkaAdminClient(bootstrap_servers="localhost:9092")

orders_topic = NewTopic(name="orders", num_partitions=3, replication_factor=1)

admin_client.create_topics(new_topics=[orders_topic], validate_only=False)

producer = KafkaProducer(bootstrap_servers='localhost:9092', value_serializer=lambda v: json.dumps(v).encode('utf-8'), key_serializer=lambda k: k.encode('utf-8'))

order_message = {"item": "book", "quantity": 2}

producer.send('orders', key='order1', value=order_message)
producer.flush()

consumer = KafkaConsumer('orders', bootstrap_servers='localhost:9092', auto_offset_reset='earliest', group_id='order_group', value_deserializer=lambda v: json.loads(v.decode('utf-8')), key_deserializer=lambda k: k.decode('utf-8'))

processed_orders = []

for message in consumer:
    processed_orders.append({'key': message.key, 'value': message.value})
    break

# Your code here to print processed_orders

Need a hint?

Use print(processed_orders) to show the list of processed orders.