Partition count strategy in Kafka - Mini Project: Build & Apply

Partition Count Strategy

📖 Scenario: You are setting up a Kafka topic for a small online store. You want to decide how many partitions the topic should have to handle the store's order messages efficiently.

🎯 Goal: Build a simple program that sets the number of partitions for a Kafka topic based on the number of expected orders per second.

📋 What You'll Learn

Create a variable with the expected number of orders per second

Create a variable for the maximum orders per partition per second

Calculate the number of partitions needed using integer division and rounding up

Print the number of partitions

💡 Why This Matters

🌍 Real World

Kafka topics use partitions to spread data load. Knowing how many partitions to create helps keep your system fast and reliable.

💼 Career

Understanding partition count strategy is important for roles like data engineers and backend developers working with Kafka.

1

Set expected orders per second

Create a variable called orders_per_second and set it to 1200.

Kafka

# Create a variable called orders_per_second and set it to 1200
# Your code here

Need a hint?

Think about how many orders your store expects to receive every second.

2

Set max orders per partition

Create a variable called max_orders_per_partition and set it to 300.

Kafka

orders_per_second = 1200
# Create a variable called max_orders_per_partition and set it to 300
# Your code here

Need a hint?

This is the maximum number of orders one partition can handle per second.

3

Calculate number of partitions

Create a variable called partitions that calculates the number of partitions needed by dividing orders_per_second by max_orders_per_partition and rounding up to the nearest whole number.

Kafka

orders_per_second = 1200
max_orders_per_partition = 300
# Calculate partitions needed and assign to partitions variable
# Your code here

Need a hint?

Use integer division and add max_orders_per_partition - 1 before dividing to round up.

4

Print the number of partitions

Write a print statement to display the value of partitions.

Kafka

orders_per_second = 1200
max_orders_per_partition = 300
partitions = (orders_per_second + max_orders_per_partition - 1) // max_orders_per_partition
# Print the number of partitions
# Your code here

Need a hint?

Use print(partitions) to show the result.