Broker configuration basics in Kafka - Mini Project: Build & Apply

Broker configuration basics

📖 Scenario: You are setting up a Kafka broker for a small company that wants to manage message streams efficiently. You need to configure the broker with basic settings to get it running properly.

🎯 Goal: Configure a Kafka broker by creating a configuration dictionary, setting a key configuration value, applying the configuration, and then printing the final broker settings.

📋 What You'll Learn

Create a dictionary called broker_config with initial settings

Add a configuration key log.retention.hours with value 168

Apply the configuration by updating the dictionary with a new key num.network.threads set to 3

Print the final broker_config dictionary

💡 Why This Matters

🌍 Real World

Kafka brokers need configuration to control how they store logs, connect to Zookeeper, and manage network threads. This project shows how to set these configurations programmatically.

💼 Career

Understanding broker configuration is important for roles like DevOps engineers, backend developers, and data engineers who manage Kafka clusters.

1

Create initial broker configuration dictionary

Create a dictionary called broker_config with these exact entries: 'broker.id': 1, 'log.dirs': '/tmp/kafka-logs', and 'zookeeper.connect': 'localhost:2181'.

Kafka

# Create the initial broker_config dictionary
# Your code here

Need a hint?

Use curly braces {} to create a dictionary with the exact keys and values.

2

Add log retention configuration

Add a new key 'log.retention.hours' with the value 168 to the existing broker_config dictionary.

Kafka

broker_config = {
    'broker.id': 1,
    'log.dirs': '/tmp/kafka-logs',
    'zookeeper.connect': 'localhost:2181'
}
# Add 'log.retention.hours' key with value 168
# Your code here

Need a hint?

Use square brackets [] to add a new key-value pair to the dictionary.

3

Apply network threads configuration

Add a new key 'num.network.threads' with the value 3 to the broker_config dictionary to apply the network threads setting.

Kafka

broker_config = {
    'broker.id': 1,
    'log.dirs': '/tmp/kafka-logs',
    'zookeeper.connect': 'localhost:2181'
}
broker_config['log.retention.hours'] = 168
# Add 'num.network.threads' key with value 3
# Your code here

Need a hint?

Use the same method as before to add another key-value pair.

4

Print the final broker configuration

Write a print statement to display the broker_config dictionary.

Kafka

broker_config = {
    'broker.id': 1,
    'log.dirs': '/tmp/kafka-logs',
    'zookeeper.connect': 'localhost:2181'
}
broker_config['log.retention.hours'] = 168
broker_config['num.network.threads'] = 3
# Print the broker_config dictionary
# Your code here

Need a hint?

Use print(broker_config) to show the dictionary.