Partition count strategy in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Kafka partition assignment code?

Consider a Kafka topic with 3 partitions and 2 consumers in a consumer group. Given the following partition assignment logic, what partitions will each consumer receive?

Kafka

partitions = [0, 1, 2]
consumers = ['consumer1', 'consumer2']
assignment = {c: [] for c in consumers}
for i, p in enumerate(partitions):
    assignment[consumers[i % len(consumers)]].append(p)
print(assignment)

A {'consumer1': [0], 'consumer2': [1, 2]}

B {'consumer1': [1, 2], 'consumer2': [0]}

C {'consumer1': [0, 2], 'consumer2': [1]}

D {'consumer1': [0, 1, 2], 'consumer2': []}

💡 Hint

Partitions are assigned in a round-robin way based on the index modulo the number of consumers.

The code assigns partitions 0 and 2 to consumer1 because 0 % 2 = 0 and 2 % 2 = 0, and partition 1 to consumer2 because 1 % 2 = 1.

🧠 Conceptual

intermediate

How does increasing partition count affect Kafka topic throughput?

Which statement best describes the effect of increasing the number of partitions in a Kafka topic on throughput?

A Increasing partitions always decreases throughput due to overhead.

B Increasing partitions can increase throughput by allowing more parallelism.

C Partition count has no effect on throughput.

D Increasing partitions reduces throughput because consumers get fewer messages.

💡 Hint

Think about how partitions allow multiple consumers to read in parallel.

More partitions allow more consumers to read in parallel, increasing throughput, but too many partitions can add overhead.

🔧 Debug

advanced

Identify the error in this partition assignment code snippet

What error will this Kafka partition assignment code raise?

Kafka

partitions = [0, 1, 2]
consumers = ['c1', 'c2']
assignment = {c: [] for c in consumers}
for i in range(len(partitions)+1):
    assignment[consumers[i % len(consumers)]].append(partitions[i])
print(assignment)

A IndexError: list index out of range

B KeyError: 'c3'

C TypeError: unsupported operand type(s)

D No error, prints assignment correctly

💡 Hint

Check the loop range and how it accesses the partitions list.

The loop runs one iteration too many (len(partitions)+1), causing partitions[i] to access an invalid index.

📝 Syntax

advanced

Which option correctly defines a Kafka topic with 5 partitions using Kafka CLI?

Choose the correct Kafka CLI command to create a topic named 'logs' with 5 partitions.

A kafka-topics.sh --create --topic logs --partitions-count 5 --bootstrap-server localhost:9092

B kafka-topics.sh --create --topic logs --partition-count 5 --bootstrap-server localhost:9092

C kafka-topics.sh --create --topic logs --num-partitions 5 --bootstrap-server localhost:9092

D kafka-topics.sh --create --topic logs --partitions 5 --bootstrap-server localhost:9092

💡 Hint

Check the exact option name for partitions in Kafka CLI.

The correct option to specify partitions is '--partitions'. Other options are invalid and cause errors.

🚀 Application

expert

Calculate the number of partitions needed for a target throughput

You want to achieve a throughput of 1 million messages per second. Each partition can handle 100,000 messages per second. How many partitions should your Kafka topic have at minimum?

A 10

B 100

C 1,000

D 100,000

💡 Hint

Divide the target throughput by the capacity per partition.

1,000,000 / 100,000 = 10 partitions needed to meet the throughput.

Practice

1. What does the partition count in a Kafka topic control?

easy

A. The number of parts a topic is divided into for parallel processing

B. The number of messages a topic can hold

C. The size limit of each message in the topic

D. The number of consumers allowed to read the topic

Solution

1. Step 1: Understand Kafka topic structure

   A Kafka topic is divided into partitions to allow parallel processing of messages.

2. Step 2: Identify what partition count controls

   The partition count sets how many parts the topic is split into, enabling multiple consumers to read in parallel.

3. Final Answer:

   The number of parts a topic is divided into for parallel processing -> Option A

4. Quick Check:

   Partition count = number of topic parts [OK]

Hint: Partition count means how many parts a topic has [OK]

Common Mistakes:

• Confusing partition count with message size
• Thinking partition count limits consumer numbers
• Assuming partition count controls message retention

2. Which of the following is the correct way to set the partition count when creating a Kafka topic using the command line?

easy

A. kafka-topics --create --topic myTopic --partitions 3 --bootstrap-server localhost:9092

B. kafka-topics --create --topic myTopic --num-partitions 3 --bootstrap-server localhost:9092

C. kafka-topics --create --topic myTopic --partition-count 3 --bootstrap-server localhost:9092

D. kafka-topics --create --topic myTopic --part 3 --bootstrap-server localhost:9092

Solution

1. Step 1: Recall Kafka topic creation syntax

   The correct flag to set partitions is --partitions.

2. Step 2: Match the correct command option

   kafka-topics --create --topic myTopic --partitions 3 --bootstrap-server localhost:9092 uses --partitions 3, which is the correct syntax.

3. Final Answer:

   kafka-topics --create --topic myTopic --partitions 3 --bootstrap-server localhost:9092 -> Option A

4. Quick Check:

   Partition flag = --partitions [OK]

Hint: Use --partitions to set partition count [OK]

Common Mistakes:

• Using --num-partitions instead of --partitions
• Using --partition-count which is invalid
• Using --part which is invalid

3. Given a Kafka topic with 4 partitions, what will be the partition number for a message with key hash % 4 = 2?

medium

A. Partition 0

B. Partition 1

C. Partition 2

D. Partition 3

Solution

1. Step 1: Understand partition assignment by key hash

   Kafka assigns messages to partitions by taking the key hash modulo the number of partitions.

2. Step 2: Calculate partition number

   Given key hash % 4 = 2, the message goes to partition 2 (zero-based indexing).

3. Final Answer:

   Partition 2 -> Option C

4. Quick Check:

   Key hash % partitions = partition number [OK]

Hint: Partition = key hash modulo partition count [OK]

Common Mistakes:

• Confusing modulo result with partition index
• Assuming partitions start at 1 instead of 0
• Ignoring key hash in partition calculation

4. You tried to create a Kafka topic with 0 partitions using the command: kafka-topics --create --topic test --num-partitions 0 --bootstrap-server localhost:9092. What is the likely issue?

medium

A. Kafka allows 0 partitions, so topic created with no data storage

B. Error because partition count must be at least 1

C. Topic created with default partition count of 1

D. Command ignored the partition count and created topic anyway

Solution

1. Step 1: Understand partition count constraints

   Kafka requires at least one partition per topic; zero is invalid.

2. Step 2: Identify command behavior

   Setting partitions to 0 causes an error because Kafka cannot create a topic without partitions.

3. Final Answer:

   Error because partition count must be at least 1 -> Option B

4. Quick Check:

   Partition count ≥ 1 required [OK]

Hint: Partition count cannot be zero or negative [OK]

Common Mistakes:

• Assuming Kafka defaults to 1 partition silently
• Thinking zero partitions means no data but topic exists
• Ignoring error messages from Kafka CLI

5. You have a Kafka topic with 5 partitions and want to increase parallel processing. Which is the best strategy?

hard

A. Delete the topic and create a new one with 1 partition

B. Decrease the partition count to 3 to reduce complexity

C. Keep partition count at 5 and add more consumers than partitions

D. Increase the partition count to 10 and rebalance consumers

Solution

1. Step 1: Understand effect of partition count on parallelism

   More partitions allow more consumers to read in parallel, increasing throughput.

2. Step 2: Evaluate options for increasing parallel processing

   Increasing partitions to 10 allows better parallelism; rebalance consumers to use new partitions.

3. Final Answer:

   Increase the partition count to 10 and rebalance consumers -> Option D

4. Quick Check:

   More partitions = better parallel processing [OK]

Hint: More partitions mean more parallel consumers [OK]

Common Mistakes:

• Reducing partitions thinking it improves speed
• Adding more consumers than partitions without increasing partitions
• Deleting topic unnecessarily