Consumer group concept in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Kafka consumer group example?

Consider a Kafka topic with 3 partitions and a consumer group with 2 consumers. Each consumer reads messages from assigned partitions. If the topic has 9 messages evenly distributed, how many messages will each consumer receive?

Kafka

Topic partitions: 3
Messages per partition: 3
Consumers in group: 2

# Messages are evenly distributed across partitions.
# Consumers get assigned partitions evenly or as balanced as possible.

A Consumer 1: 4 messages, Consumer 2: 5 messages

B Consumer 1: 3 messages, Consumer 2: 6 messages

C Consumer 1: 5 messages, Consumer 2: 4 messages

D Consumer 1: 6 messages, Consumer 2: 3 messages

💡 Hint

Partitions are assigned to consumers. Each partition's messages go to one consumer only.

With 3 partitions and 2 consumers, one consumer gets 2 partitions (6 messages) and the other gets 1 partition (3 messages). So one consumer reads 6 messages, the other 3.

🧠 Conceptual

intermediate

What happens when a new consumer joins an existing consumer group?

In Kafka, when a new consumer joins a consumer group that is already consuming from a topic, what is the expected behavior?

A The new consumer is ignored until the current consumers disconnect.

B The new consumer starts consuming from the last committed offset without affecting others.

C The group triggers a rebalance and partitions are reassigned among all consumers.

D The new consumer consumes from all partitions alongside existing consumers.

💡 Hint

Think about how Kafka ensures each partition is consumed by only one consumer in the group.

When a new consumer joins, Kafka triggers a rebalance to redistribute partitions evenly among all consumers in the group.

🔧 Debug

advanced

Why does Consumer 3 not receive any messages?

Given a topic with 2 partitions and a consumer group with 3 consumers, one consumer is not receiving any messages. Here is the consumer group assignment log snippet:

Consumer 1 assigned partitions: 0, 1
Consumer 2 assigned partitions: 
Consumer 3 assigned partitions: 

Why is Consumer 3 not receiving messages?

A Because the number of consumers exceeds the number of partitions, one consumer gets no partitions.

B Because Consumer 3 has a network issue and cannot receive messages.

C Because Kafka assigns partitions randomly and Consumer 3 was skipped by mistake.

D Because Consumer 3 is not subscribed to the topic.

💡 Hint

Consider how Kafka assigns partitions when consumers outnumber partitions.

Kafka assigns partitions so that each partition is consumed by only one consumer. If there are more consumers than partitions, some consumers get no partitions.

📝 Syntax

advanced

What error does this Kafka consumer group code produce?

Consider this Kafka consumer group code snippet in Java:

Properties props = new Properties();
props.put("group.id", "my-group");
props.put("bootstrap.servers", "localhost:9092");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

KafkaConsumer consumer = new KafkaConsumer<>(props);
consumer.subscribe(Arrays.asList("my-topic"));

while(true) {
    ConsumerRecords records = consumer.poll(Duration.ofMillis(100));
    for (ConsumerRecord record : records) {
        System.out.println(record.value());
    }
}
consumer.close();

What error will this code produce?

A Compilation error because consumer.close() is unreachable after infinite loop.

B NullPointerException because props is missing required properties.

C Compilation error due to missing import statements.

D No error; code runs and consumes messages correctly.

💡 Hint

Look at the infinite loop and the position of consumer.close().

The infinite while(true) loop makes consumer.close() unreachable, causing a compilation error (unreachable statement).

🚀 Application

expert

How many partitions will each consumer get after rebalance?

A Kafka topic has 7 partitions. A consumer group has 4 consumers. After a rebalance, how many partitions will each consumer be assigned?

A Consumers 1-2 get 3 partitions each, Consumers 3-4 get 0 partitions.

B Consumers 1-3 get 2 partitions each, Consumer 4 gets 1 partition.

C Each consumer gets exactly 1 partition, and 3 partitions remain unassigned.

D Consumers 1-4 get 1 partition each, and 3 partitions are assigned to a new consumer.

💡 Hint

Partitions are distributed as evenly as possible among consumers.

7 partitions divided among 4 consumers means three consumers get 2 partitions each (2*3=6) and one consumer gets 1 partition (1*1=1), totaling 7.

Practice

1. What is the main purpose of a consumer group in Kafka?

easy

A. To allow multiple consumers to share the work of reading messages

B. To store messages permanently in Kafka

C. To produce messages to Kafka topics

D. To encrypt messages during transmission

Solution

1. Step 1: Understand consumer group role

   A consumer group lets multiple consumers work together to read messages from a topic.

2. Step 2: Compare other options

   Storing messages, producing messages, and encrypting are not roles of consumer groups.

3. Final Answer:

   To allow multiple consumers to share the work of reading messages -> Option A

4. Quick Check:

   Consumer group = share reading work [OK]

Hint: Consumer groups split message reading among consumers [OK]

Common Mistakes:

• Confusing consumer groups with producers
• Thinking consumer groups store messages
• Assuming consumer groups encrypt data

2. Which of the following is the correct way to specify a consumer group ID in Kafka consumer configuration?

easy

A. "groupName" : "my-group"

B. "group.id" : "my-group"

C. "consumer.group" : "my-group"

D. "consumerId" : "my-group"

Solution

1. Step 1: Recall Kafka consumer config keys

   The correct key to set the consumer group is "group.id".

2. Step 2: Check other options

   Other keys like "consumer.group", "groupName", or "consumerId" are incorrect or do not exist.

3. Final Answer:

   "group.id" : "my-group" -> Option B

4. Quick Check:

   Consumer group ID key = "group.id" [OK]

Hint: Remember Kafka uses "group.id" for consumer groups [OK]

Common Mistakes:

• Using incorrect config keys
• Confusing consumer ID with group ID
• Misspelling the key name

3. Given a Kafka topic with 4 partitions and a consumer group with 2 consumers, how many partitions will each consumer read from?

medium

A. Each consumer reads from 2 partitions

B. Each consumer reads from 4 partitions

C. Each consumer reads from 1 partition

D. Each consumer reads from 0 partitions

Solution

1. Step 1: Understand partition assignment

   Partitions are divided among consumers in a group. With 4 partitions and 2 consumers, partitions split evenly.

2. Step 2: Calculate partitions per consumer

   4 partitions / 2 consumers = 2 partitions per consumer.

3. Final Answer:

   Each consumer reads from 2 partitions -> Option A

4. Quick Check:

   Partitions per consumer = total partitions / consumers = 2 [OK]

Hint: Partitions divide evenly among consumers in a group [OK]

Common Mistakes:

• Assuming each consumer reads all partitions
• Thinking partitions are not shared
• Ignoring partition count

4. What is wrong with this Kafka consumer group code snippet?

props.put("group.id", "my-group");
props.put("enable.auto.commit", "false");
consumer.subscribe(Arrays.asList("my-topic"));
consumer.poll(Duration.ofMillis(100));
// Missing commit after poll

medium

A. The poll timeout is too long

B. The group.id is incorrectly set

C. The consumer subscribes to the wrong topic

D. The consumer never commits offsets, risking message reprocessing

Solution

1. Step 1: Analyze auto commit setting

   Auto commit is disabled, so offsets must be committed manually.

2. Step 2: Check code for commit call

   Code calls poll but does not commit offsets, so messages may be reprocessed on restart.

3. Final Answer:

   The consumer never commits offsets, risking message reprocessing -> Option D

4. Quick Check:

   Auto commit false needs manual commit [OK]

Hint: Disable auto commit means manual commit needed [OK]

Common Mistakes:

• Forgetting to commit offsets manually
• Assuming auto commit is enabled by default
• Ignoring poll timeout relevance

5. You have a Kafka topic with 3 partitions and a consumer group with 4 consumers. What happens to the extra consumer?

hard

A. The extra consumer causes an error in the group

B. The extra consumer reads from all partitions

C. The extra consumer remains idle and does not receive messages

D. Kafka creates a new partition for the extra consumer

Solution

1. Step 1: Understand partition to consumer assignment

   Each partition can be assigned to only one consumer in the group.

2. Step 2: Analyze consumer count vs partitions

   With 3 partitions and 4 consumers, only 3 consumers get partitions; the 4th consumer stays idle.

3. Final Answer:

   The extra consumer remains idle and does not receive messages -> Option C

4. Quick Check:

   More consumers than partitions means idle consumers [OK]

Hint: Consumers > partitions means some consumers idle [OK]

Common Mistakes:

• Thinking Kafka creates new partitions automatically
• Assuming extra consumers share partitions
• Believing extra consumers cause errors