Resource planning and capacity in Kafka - Mini Project: Build & Apply

Resource Planning and Capacity with Kafka

📖 Scenario: You work in a company that uses Apache Kafka to handle messages between different services. You want to plan how many partitions each topic should have to handle the expected load without delays.

🎯 Goal: Build a simple Kafka resource planner that calculates the total capacity needed based on the number of topics and partitions, and then outputs the total capacity.

📋 What You'll Learn

Create a dictionary called topics with topic names as keys and number of partitions as values

Create a variable called capacity_per_partition that holds the capacity value per partition

Calculate the total capacity by multiplying partitions by capacity per partition for all topics using a comprehension

Print the total capacity with a descriptive message

💡 Why This Matters

🌍 Real World

Kafka is widely used for messaging in distributed systems. Planning partitions and capacity helps avoid bottlenecks and ensures smooth data flow.

💼 Career

Understanding resource planning in Kafka is important for roles like DevOps engineers, backend developers, and system architects who manage scalable data pipelines.

1

Create the topics dictionary

Create a dictionary called topics with these exact entries: 'orders': 3, 'payments': 2, 'shipments': 4

Kafka

# Create the topics dictionary with exact entries
# Your code here

Need a hint?

Use curly braces to create a dictionary with keys and values separated by colons.

2

Set capacity per partition

Create a variable called capacity_per_partition and set it to 1000 to represent capacity units per partition

Kafka

topics = {'orders': 3, 'payments': 2, 'shipments': 4}
# Set capacity_per_partition to 1000
# Your code here

Need a hint?

Just assign the number 1000 to the variable capacity_per_partition.

3

Calculate total capacity

Create a variable called total_capacity that calculates the sum of partitions times capacity_per_partition for all topics using a comprehension with for partitions in topics.values()

Kafka

topics = {'orders': 3, 'payments': 2, 'shipments': 4}
capacity_per_partition = 1000
# Calculate total_capacity using a comprehension
# Your code here

Need a hint?

Use sum() with a generator expression multiplying each partitions value by capacity_per_partition.

4

Print the total capacity

Write a print statement that outputs: Total capacity needed: {total_capacity} using an f-string

Kafka

topics = {'orders': 3, 'payments': 2, 'shipments': 4}
capacity_per_partition = 1000
total_capacity = sum(partitions * capacity_per_partition for partitions in topics.values())
# Print the total capacity using an f-string
# Your code here

Need a hint?

Use print() with an f-string to show the total_capacity variable.