Why producers publish data in Kafka - See It in Action

Why Producers Publish Data in Kafka

📖 Scenario: Imagine you run a small online store. You want to keep track of every order your customers make. To do this, you use Kafka, a system that helps send and store messages quickly and safely.

🎯 Goal: Learn why producers publish data to Kafka and how to create a simple producer that sends order messages.

📋 What You'll Learn

Create a Kafka topic named orders

Write a producer that sends order data to the orders topic

Understand the role of the producer in sending data

Print confirmation after sending each order

💡 Why This Matters

🌍 Real World

Online stores, banks, and many apps use Kafka producers to send data like orders, transactions, or logs quickly and reliably.

💼 Career

Understanding Kafka producers is important for roles in data engineering, backend development, and real-time data processing.

1

Create a list of orders

Create a list called orders with these exact dictionaries: {'order_id': 101, 'item': 'Book'}, {'order_id': 102, 'item': 'Pen'}, {'order_id': 103, 'item': 'Notebook'}

Kafka

# Create the list of orders
# Your code here

Need a hint?

Use a list with dictionaries inside. Each dictionary has keys 'order_id' and 'item'.

2

Set the Kafka topic name

Create a variable called topic_name and set it to the string 'orders'

Kafka

orders = [{'order_id': 101, 'item': 'Book'}, {'order_id': 102, 'item': 'Pen'}, {'order_id': 103, 'item': 'Notebook'}]
# Set the Kafka topic name
# Your code here

Need a hint?

Just assign the string 'orders' to the variable topic_name.

3

Send orders to Kafka topic

Use a for loop with variable order to go through orders and inside the loop, call producer.send(topic_name, value=order)

Kafka

orders = [{'order_id': 101, 'item': 'Book'}, {'order_id': 102, 'item': 'Pen'}, {'order_id': 103, 'item': 'Notebook'}]
topic_name = 'orders'

from kafka import KafkaProducer
import json

producer = KafkaProducer(value_serializer=lambda v: json.dumps(v).encode('utf-8'))

# Send each order to the Kafka topic
# Your code here

Need a hint?

Use a for loop to send each order dictionary to Kafka using producer.send().

4

Print confirmation after sending

Inside the for loop, after sending, write print(f"Sent order {order['order_id']} to topic {topic_name}")

Kafka

orders = [{'order_id': 101, 'item': 'Book'}, {'order_id': 102, 'item': 'Pen'}, {'order_id': 103, 'item': 'Notebook'}]
topic_name = 'orders'

from kafka import KafkaProducer
import json

producer = KafkaProducer(value_serializer=lambda v: json.dumps(v).encode('utf-8'))

for order in orders:
    producer.send(topic_name, value=order)
    # Print confirmation
    # Your code here

Need a hint?

Use a print statement inside the loop to show which order was sent and to which topic.