Client metrics monitoring in Kafka - Mini Project: Build & Apply

Client metrics monitoring

📖 Scenario: You work in a company that collects client metrics data through Kafka topics. Your task is to create a simple Kafka consumer that reads client metrics from a topic and processes them step-by-step.

🎯 Goal: Build a Kafka consumer that subscribes to a topic named client_metrics, filters metrics based on a threshold, and prints the filtered metrics.

📋 What You'll Learn

Create a Kafka consumer configuration dictionary with the required settings

Set a threshold variable to filter client metrics

Consume messages from the client_metrics topic and filter metrics above the threshold

Print the filtered metrics

💡 Why This Matters

🌍 Real World

Monitoring client metrics helps companies track performance and detect issues early by processing data streams in real time.

💼 Career

Kafka is widely used in DevOps and data engineering roles to handle streaming data pipelines and real-time analytics.

1

Create Kafka consumer configuration

Create a dictionary called consumer_config with these exact entries: 'bootstrap.servers': 'localhost:9092', 'group.id': 'metrics_group', and 'auto.offset.reset': 'earliest'.

Kafka

# Create the Kafka consumer configuration dictionary
# Your code here

Need a hint?

Use a Python dictionary with the exact keys and values as specified.

2

Set the metric threshold

Create a variable called threshold and set it to the integer 50.

Kafka

consumer_config = {
    'bootstrap.servers': 'localhost:9092',
    'group.id': 'metrics_group',
    'auto.offset.reset': 'earliest'
}
# Set the threshold value to 50
# Your code here

Need a hint?

Just assign the number 50 to the variable named threshold.

3

Filter client metrics above threshold

Create a list called filtered_metrics that contains only the values from the list metrics which are greater than the variable threshold. Use a list comprehension.

Kafka

consumer_config = {
    'bootstrap.servers': 'localhost:9092',
    'group.id': 'metrics_group',
    'auto.offset.reset': 'earliest'
}

threshold = 50

metrics = [45, 67, 23, 89, 50, 77]
# Create filtered_metrics list with values greater than threshold
# Your code here

Need a hint?

Use a list comprehension with the condition m > threshold.

4

Print the filtered metrics

Write a print statement to display the variable filtered_metrics.

Kafka

consumer_config = {
    'bootstrap.servers': 'localhost:9092',
    'group.id': 'metrics_group',
    'auto.offset.reset': 'earliest'
}

threshold = 50

metrics = [45, 67, 23, 89, 50, 77]

filtered_metrics = [m for m in metrics if m > threshold]
# Print the filtered_metrics list
# Your code here

Need a hint?

Use print(filtered_metrics) to show the filtered list.