Consumer configuration in Kafka - Mini Project: Build & Apply

Kafka Consumer Configuration

📖 Scenario: You are building a simple Kafka consumer application that reads messages from a Kafka topic. To do this, you need to set up the consumer configuration correctly.

🎯 Goal: Create a Kafka consumer configuration dictionary with specific settings, then use it to create a consumer and print the configuration to verify it.

📋 What You'll Learn

Create a dictionary called consumer_config with exact key-value pairs

Add a variable called group_id with a specific string value

Create a Kafka consumer using consumer_config and group_id

Print the consumer_config dictionary

💡 Why This Matters

🌍 Real World

Kafka consumers are used in real applications to read and process streams of data from Kafka topics, such as logs, user activity, or sensor data.

💼 Career

Understanding how to configure Kafka consumers is essential for roles in data engineering, backend development, and real-time data processing.

1

Create the initial consumer configuration dictionary

Create a dictionary called consumer_config with these exact entries: 'bootstrap.servers': 'localhost:9092', 'auto.offset.reset': 'earliest', and 'enable.auto.commit': False.

Kafka

# Create the consumer_config dictionary with the required entries
# Your code here

Need a hint?

Use curly braces {} to create a dictionary and separate key-value pairs with commas.

2

Add the consumer group ID configuration

Create a variable called group_id and set it to the string 'my-consumer-group'.

Kafka

consumer_config = {
    'bootstrap.servers': 'localhost:9092',
    'auto.offset.reset': 'earliest',
    'enable.auto.commit': False
}
# Create the group_id variable with the exact string value
# Your code here

Need a hint?

Assign the string 'my-consumer-group' to the variable group_id.

3

Add the group ID to the consumer configuration

Add the key 'group.id' with the value of the variable group_id to the consumer_config dictionary.

Kafka

consumer_config = {
    'bootstrap.servers': 'localhost:9092',
    'auto.offset.reset': 'earliest',
    'enable.auto.commit': False
}
group_id = 'my-consumer-group'
# Add 'group.id' key with value group_id to consumer_config
# Your code here

Need a hint?

Use square brackets [] to add a new key-value pair to the dictionary.

4

Print the consumer configuration

Write a print statement to display the consumer_config dictionary.

Kafka

consumer_config = {
    'bootstrap.servers': 'localhost:9092',
    'auto.offset.reset': 'earliest',
    'enable.auto.commit': False
}
group_id = 'my-consumer-group'
consumer_config['group.id'] = group_id
# Print the consumer_config dictionary
# Your code here

Need a hint?

Use print(consumer_config) to show the dictionary content.