Message key and value in Kafka - Mini Project: Build & Apply

Message key and value

📖 Scenario: You are working with Apache Kafka to send messages to a topic. Each message has a key and a value. The key helps Kafka decide which partition the message goes to, and the value is the actual data you want to send.Imagine you are sending user login events where the key is the user ID and the value is the login timestamp.

🎯 Goal: Build a simple Kafka producer that sends messages with a key and a value to a topic called user-logins.

📋 What You'll Learn

Create a Kafka producer configuration

Send messages with a key and a value

Use the topic name user-logins

Print confirmation after sending each message

💡 Why This Matters

🌍 Real World

Kafka is widely used for real-time data streaming. Sending messages with keys helps organize data by partitions, which improves performance and scalability.

💼 Career

Understanding how to send messages with keys and values is essential for roles involving data engineering, backend development, and real-time analytics.

1

Set up Kafka producer properties

Create a dictionary called producer_props with 'bootstrap_servers': 'localhost:9092'.

Kafka

# Create the producer properties dictionary
# Your code here

Need a hint?

Use a Python dictionary with the exact keys and values for Kafka producer configuration.

2

Create the Kafka producer instance

Create a variable called producer and assign it a new Kafka producer instance using producer_props.

Kafka

producer_props = {
    'bootstrap_servers': 'localhost:9092'
}
# Create the Kafka producer instance
# Your code here

Need a hint?

Use KafkaProducer with the unpacked producer_props dictionary.

3

Send messages with key and value

Use producer.send to send a message to the topic 'user-logins' with the key b'user123' and the value b'2024-06-01T10:00:00Z'.

Kafka

producer_props = {
    'bootstrap_servers': 'localhost:9092'
}
producer = KafkaProducer(**producer_props)
# Send a message with key and value
# Your code here

Need a hint?

Use producer.send with the exact topic, key, and value bytes.

4

Print confirmation after sending

Write a print statement that outputs exactly "Sent message with key user123 to topic user-logins".

Kafka

producer_props = {
    'bootstrap_servers': 'localhost:9092'
}
producer = KafkaProducer(**producer_props)
producer.send('user-logins', key=b'user123', value=b'2024-06-01T10:00:00Z')
# Print confirmation message
# Your code here

Need a hint?

Use print with the exact message text.