Resource planning and capacity in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Partition assignment output in Kafka consumer group

What is the output of the following Kafka consumer group partition assignment code snippet?

Kafka

from kafka import KafkaConsumer

consumer = KafkaConsumer(
    'topic1',
    group_id='group1',
    bootstrap_servers=['localhost:9092'],
    enable_auto_commit=False
)

partitions = consumer.partitions_for_topic('topic1')
print(sorted(partitions))

A None

B [0, 1, 2]

C set()

D ['partition0', 'partition1', 'partition2']

💡 Hint

Check what partitions_for_topic returns and how to convert it to a list.

The method partitions_for_topic returns a set of partition numbers. Sorting it and printing shows a list of partition numbers like [0, 1, 2].

❓ Predict Output

intermediate

Calculating consumer lag from offsets

Given the following code to calculate consumer lag, what is the value of `lag` after execution?

Kafka

end_offsets = {0: 150, 1: 200}
committed_offsets = {0: 140, 1: 180}
lag = sum(end_offsets[p] - committed_offsets[p] for p in end_offsets)
print(lag)

A 50

B None

C 20

D 30

💡 Hint

Calculate the difference for each partition and sum them.

Partition 0 lag is 150 - 140 = 10, partition 1 lag is 200 - 180 = 20, total lag is 10 + 20 = 30.

🔧 Debug

advanced

Identify the error in Kafka producer throughput calculation

What error will the following code raise when calculating throughput?

Kafka

messages_sent = 1000
elapsed_time = 0
throughput = messages_sent / elapsed_time
print(f'Throughput: {throughput} msgs/sec')

A ZeroDivisionError

B TypeError

C NameError

D No error, prints 'Throughput: 1000 msgs/sec'

💡 Hint

Check the value of elapsed_time before division.

Dividing by zero causes a ZeroDivisionError in Python.

❓ Predict Output

advanced

Result of consumer assignment after rebalance

What will be the output of the following code snippet simulating consumer assignment after rebalance?

Kafka

consumer_assignment = {0: 'consumer1', 1: 'consumer2', 2: 'consumer1'}
from collections import Counter
count = Counter(consumer_assignment.values())
print(count['consumer1'])

A 2

B 1

C 3

D KeyError

💡 Hint

Count how many partitions are assigned to 'consumer1'.

'consumer1' is assigned partitions 0 and 2, so count is 2.

🧠 Conceptual

expert

Maximum throughput calculation with partition and consumer limits

Given a Kafka topic with 12 partitions and 4 consumers in a group, each consumer can process 500 messages per second. What is the maximum total throughput achievable assuming perfect load balancing?

A 6000 messages per second

B 3000 messages per second

C 2000 messages per second

D 4000 messages per second

💡 Hint

Each consumer processes 500 msgs/sec. Total throughput is consumers * per consumer rate.

With 4 consumers each processing 500 msgs/sec, total throughput is 4 * 500 = 2000 msgs/sec. But since there are 12 partitions, and each consumer can only process partitions assigned, the max throughput is limited by consumers. So total is 4 * 500 = 2000 msgs/sec.