First message (produce and consume) in Kafka - Mini Project: Build & Apply

First message (produce and consume)

📖 Scenario: You are working with Apache Kafka to send and receive messages between applications. Kafka uses topics to organize messages. In this project, you will create a simple Kafka producer to send a message and a Kafka consumer to receive it.

🎯 Goal: Build a Kafka producer that sends a message to a topic called first_topic and a Kafka consumer that reads and prints that message.

📋 What You'll Learn

Create a Kafka producer that sends the message 'Hello Kafka!' to the topic first_topic.

Create a Kafka consumer that subscribes to first_topic and reads messages.

Print the received message to the console.

💡 Why This Matters

🌍 Real World

Kafka is widely used for building real-time data pipelines and streaming apps. Sending and receiving messages is the core of Kafka's communication.

💼 Career

Understanding Kafka producers and consumers is essential for roles in data engineering, backend development, and real-time analytics.

1

Create Kafka producer and send message

Create a Kafka producer using the KafkaProducer class from kafka-python. Send the message 'Hello Kafka!' to the topic 'first_topic'. Use bootstrap_servers=['localhost:9092'] when creating the producer.

Kafka

# Your code here to create producer and send message

Need a hint?

Use KafkaProducer to create the producer. The message must be bytes, so prefix the string with b.

2

Create Kafka consumer and subscribe to topic

Create a Kafka consumer using the KafkaConsumer class from kafka-python. Subscribe the consumer to the topic 'first_topic'. Use bootstrap_servers=['localhost:9092'] when creating the consumer.

Kafka

from kafka import KafkaConsumer

# Your code here to create consumer and subscribe to 'first_topic'

Need a hint?

Use KafkaConsumer('first_topic', bootstrap_servers=['localhost:9092'], auto_offset_reset='earliest') to create and subscribe in one step.

3

Read message from consumer

Use a for loop to read messages from the consumer. Inside the loop, get the message value using message.value. Decode the bytes to a string using .decode('utf-8').

Kafka

# Your code here to read messages from consumer and decode them

Need a hint?

Use a for loop to get messages. Decode the message bytes to string.

4

Print the received message

Print the decoded message stored in the variable text using print(text).

Kafka

# Your code here to print the message

Need a hint?

Use print(text) to show the message on the screen.