Auto-scaling strategies in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Kafka consumer group scaling code?

Consider a Kafka consumer group with 3 consumers consuming from 3 partitions. The code below simulates scaling the consumers dynamically. What will be the number of active consumers after running this code?

Kafka

consumers = 3
partitions = 3

# Simulate scaling down by 1 consumer
consumers -= 1

# Simulate scaling up by 2 consumers
consumers += 2

# But max consumers cannot exceed partitions
active_consumers = min(consumers, partitions)
print(active_consumers)

A 3

B 1

C 2

D 4

💡 Hint

Remember that the number of active consumers cannot exceed the number of partitions.

The code first reduces consumers from 3 to 2, then increases to 4. But since partitions are 3, active consumers are capped at 3.

🧠 Conceptual

intermediate

Which auto-scaling metric is best for Kafka consumer groups?

You want to auto-scale Kafka consumers based on workload. Which metric is most appropriate to trigger scaling?

A Number of active partitions

B Consumer lag (messages behind)

C CPU usage of consumer machines

D Network bandwidth usage

💡 Hint

Think about what indicates consumers are falling behind.

Consumer lag shows how many messages are waiting to be processed, so scaling based on lag helps keep up with workload.

🔧 Debug

advanced

Why does this Kafka auto-scaling script fail to scale down?

This Python script tries to scale down Kafka consumers when lag is low, but consumers never decrease. What is the bug?

Kafka

consumers = 5
lag = 0

if lag < 10:
    consumers = consumers - 1
else:
    consumers = consumers + 1

print(consumers)

A The script does not check if consumers go below 1

B The condition should be lag <= 10 instead of lag < 10

C The print statement is outside the if block

D The lag variable is never updated dynamically

💡 Hint

Check if lag changes during the script execution.

Lag is set to 0 and never updated, so the condition always triggers scale down once, but in real use lag should be updated dynamically.

📝 Syntax

advanced

Which option correctly implements a Kafka consumer auto-scaling check in Python?

Select the code snippet that correctly checks if consumer lag exceeds threshold and triggers scale up.

A

if lag > threshold
    scale_up()

B

if (lag > threshold)
    scale_up()

C

if lag > threshold:
    scale_up()

D

if lag > threshold then
    scale_up()

💡 Hint

Remember Python requires a colon after if condition.

Python syntax requires a colon at the end of the if statement line.

🚀 Application

expert

How many consumers will be active after this auto-scaling logic runs?

Given 4 partitions and initial 2 consumers, the code below scales consumers based on lag values. What is the final number of active consumers?

Kafka

partitions = 4
consumers = 2
lag_values = [5, 15, 8, 20]

for lag in lag_values:
    if lag > 10:
        consumers += 1
    elif lag < 5 and consumers > 1:
        consumers -= 1

active_consumers = min(consumers, partitions)
print(active_consumers)

A 4

B 2

C 3

D 5

💡 Hint

Count how many times consumers increase or decrease, then cap by partitions.

Lag values >10 occur twice, increasing consumers from 2 to 4. Lag <5 never occurs, so no decrease. Max consumers capped at 4 partitions.