Why monitoring prevents outages in Kafka - See It in Action

Why monitoring prevents outages

📖 Scenario: You are managing a Kafka message system that processes orders for an online store. To keep the system healthy, you want to monitor the number of messages waiting in the queue. If the queue grows too large, it might mean the system is slow or stuck, which can cause outages.

🎯 Goal: Build a simple monitoring script that tracks the number of messages in a Kafka topic and alerts if the count exceeds a safe limit. This helps prevent outages by catching problems early.

📋 What You'll Learn

Create a dictionary called kafka_queues with these exact entries: 'orders': 120, 'payments': 80, 'shipments': 50

Create a variable called threshold and set it to 100

Use a for loop with variables topic and count to iterate over kafka_queues.items() and create a list alerts of topics where count is greater than threshold

Print the alerts list to show which topics need attention

💡 Why This Matters

🌍 Real World

Monitoring Kafka queues helps catch problems early, like slow consumers or stuck messages, preventing system outages.

💼 Career

DevOps engineers use monitoring scripts like this to keep message systems healthy and ensure smooth data flow in production.

1

Create Kafka queue message counts

Create a dictionary called kafka_queues with these exact entries: 'orders': 120, 'payments': 80, 'shipments': 50

Kafka

# Create the kafka_queues dictionary with exact entries
# Your code here

Need a hint?

Use curly braces to create a dictionary with keys and values exactly as shown.

2

Set the alert threshold

Create a variable called threshold and set it to 100

Kafka

kafka_queues = {'orders': 120, 'payments': 80, 'shipments': 50}
# Create the threshold variable and set it to 100
# Your code here

Need a hint?

Just assign the number 100 to the variable named threshold.

3

Find topics exceeding the threshold

Use a for loop with variables topic and count to iterate over kafka_queues.items() and create a list alerts of topics where count is greater than threshold

Kafka

kafka_queues = {'orders': 120, 'payments': 80, 'shipments': 50}
threshold = 100
# Create an alerts list with topics where count > threshold
# Your code here

Need a hint?

Use a list comprehension to filter topics where count is greater than threshold.

4

Display the alert topics

Print the alerts list to show which topics need attention

Kafka

kafka_queues = {'orders': 120, 'payments': 80, 'shipments': 50}
threshold = 100
alerts = [topic for topic, count in kafka_queues.items() if count > threshold]
# Print the alerts list
# Your code here

Need a hint?

Use print(alerts) to show the list of topics exceeding the threshold.