Static group membership in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Static group membership with fixed member IDs

Given the following Kafka consumer group configuration with static membership, what will be the output when the consumer restarts with the same member ID?

Kafka

consumerProps.put("group.id", "my-group");
consumerProps.put("group.instance.id", "consumer-1");

// Consumer subscribes and polls messages
consumer.subscribe(Collections.singletonList("my-topic"));
ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(1));
System.out.println("Polled records count: " + records.count());

// Consumer closes and restarts with same group.instance.id
consumer.close();
consumer = new KafkaConsumer<>(consumerProps);
consumer.subscribe(Collections.singletonList("my-topic"));
records = consumer.poll(Duration.ofSeconds(1));
System.out.println("Polled records count after restart: " + records.count());

A

Polled records count: 0
Polled records count after restart: 10

B

Polled records count: 10
Polled records count after restart: 10

C

Polled records count: 10
Polled records count after restart: 0

D

Polled records count: 0
Polled records count after restart: 0

💡 Hint

Static group membership allows the consumer to keep its partition assignments across restarts, avoiding rebalance delays.

When a consumer uses static group membership with a fixed group.instance.id, Kafka recognizes it as the same member after restart. This prevents rebalancing and the consumer resumes from its last committed offsets. Therefore, after restart, it will not re-poll the same messages again, resulting in zero new records polled.

🧠 Conceptual

intermediate

Understanding static group membership benefits

Which of the following is a key benefit of using static group membership in Kafka consumer groups?

A It allows consumers to dynamically change their group.instance.id during runtime.

B It automatically increases the number of partitions in the topic.

C It eliminates the need for consumers to commit offsets manually.

D It reduces consumer group rebalances when a consumer restarts with the same member ID.

💡 Hint

Think about what happens when a consumer leaves and rejoins the group with the same ID.

Static group membership assigns a fixed member ID to a consumer. When the consumer restarts with the same ID, Kafka treats it as the same member, avoiding unnecessary rebalances and improving stability.

🔧 Debug

advanced

Debugging static group membership error

A Kafka consumer configured with static group membership fails to join the group and throws an error: "Member ID consumer-1 is already in use." What is the most likely cause?

A The consumer did not commit offsets before restarting.

B Two consumers are running simultaneously with the same group.instance.id.

C The topic has fewer partitions than consumers.

D The consumer group ID is missing in the configuration.

💡 Hint

Static group membership requires unique member IDs per active consumer.

Static group membership requires each active consumer to have a unique group.instance.id. If two consumers use the same ID at the same time, Kafka rejects the second with a member ID conflict error.

❓ Predict Output

advanced

Effect of static group membership on rebalance listeners

Consider a Kafka consumer with a RebalanceListener that logs partition assignments. The consumer uses static group membership and restarts with the same group.instance.id. What will the listener output after restart?

Kafka

consumerProps.put("group.id", "my-group");
consumerProps.put("group.instance.id", "consumer-42");

consumer.subscribe(Collections.singletonList("my-topic"), new ConsumerRebalanceListener() {
    public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
        System.out.println("Partitions revoked: " + partitions);
    }
    public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
        System.out.println("Partitions assigned: " + partitions);
    }
});

consumer.poll(Duration.ofSeconds(1));
consumer.close();

// Restart consumer with same group.instance.id
consumer = new KafkaConsumer<>(consumerProps);
consumer.subscribe(Collections.singletonList("my-topic"), new ConsumerRebalanceListener() {
    public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
        System.out.println("Partitions revoked: " + partitions);
    }
    public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
        System.out.println("Partitions assigned: " + partitions);
    }
});
consumer.poll(Duration.ofSeconds(1));

A

Partitions revoked: []
Partitions assigned: [my-topic-0, my-topic-1]

B

Partitions revoked: [my-topic-0, my-topic-1]
Partitions assigned: [my-topic-0, my-topic-1]

C

Partitions revoked: []
Partitions assigned: []

D No output from rebalance listener after restart

💡 Hint

Static membership avoids unnecessary rebalances on restart.

With static group membership, the consumer is recognized as the same member after restart. Kafka does not revoke partitions before restart and reassigns them immediately, so revoked partitions list is empty and assigned partitions are the same as before.

🧠 Conceptual

expert

Static group membership and offset retention

How does static group membership affect the retention of consumer group offsets in Kafka when a consumer leaves the group unexpectedly?

A Offsets are retained longer because Kafka waits for the static member to rejoin before removing offsets.

B Offsets are immediately deleted when the consumer leaves, regardless of static membership.

C Offsets are never deleted for static members, even if they never rejoin.

D Static group membership has no effect on offset retention policies.

💡 Hint

Think about how Kafka treats static members differently from dynamic members on unexpected leave.

Kafka retains offsets for static group members longer because it expects the member to rejoin with the same group.instance.id. This reduces unnecessary offset deletions and rebalances.

Practice

1.

What is the main purpose of using group.instance.id in Kafka consumer groups?

easy

A. To increase the number of partitions in a topic

B. To encrypt messages within the consumer group

C. To automatically commit offsets more frequently

D. To assign a stable identity to a consumer and enable static group membership

Solution

1. Step 1: Understand the role of group.instance.id

   This configuration assigns a unique and stable identity to a Kafka consumer within a group.

2. Step 2: Connect stable identity to static group membership

   With a stable identity, Kafka can avoid unnecessary rebalances when a consumer restarts briefly, improving stability.

3. Final Answer:

   To assign a stable identity to a consumer and enable static group membership -> Option D

4. Quick Check:

   Stable consumer identity = static group membership [OK]

Hint: Static membership means stable consumer ID to avoid rebalances [OK]

Common Mistakes:

• Confusing group.instance.id with partition count
• Thinking it controls offset commit frequency
• Assuming it encrypts messages

2.

Which of the following is the correct way to set group.instance.id in a Kafka consumer configuration?

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
// What is the correct line to set static membership?

easy

A. props.put("group.instance.id", "consumer-1");

B. props.put("group.id.instance", "consumer-1");

C. props.put("static.group.id", "consumer-1");

D. props.put("group.instance", "consumer-1");

Solution

1. Step 1: Recall the exact property name for static membership

   The correct property to set a static consumer ID is group.instance.id.

2. Step 2: Match the correct syntax in Java Properties

   Using props.put("group.instance.id", "consumer-1") correctly sets the static ID.

3. Final Answer:

   props.put("group.instance.id", "consumer-1"); -> Option A

4. Quick Check:

   Correct property name = group.instance.id [OK]

Hint: Remember exact property name: group.instance.id [OK]

Common Mistakes:

• Swapping words in the property name
• Using incomplete or incorrect property keys
• Confusing with group.id

3.

Consider this Kafka consumer group with static membership enabled using group.instance.id. If a consumer with group.instance.id = "consumer-1" disconnects briefly and reconnects, what happens?

medium

A. The consumer is removed permanently from the group

B. The group triggers a full rebalance and reassigns partitions

C. The consumer resumes without triggering a rebalance

D. The consumer must use a new group.instance.id to reconnect

Solution

1. Step 1: Understand static membership behavior on brief disconnect

   Static membership keeps the consumer identity stable, so brief disconnects do not cause rebalances.

2. Step 2: Identify the effect on partition assignment

   The consumer resumes with the same partitions without triggering a rebalance, improving latency and stability.

3. Final Answer:

   The consumer resumes without triggering a rebalance -> Option C

4. Quick Check:

   Static membership avoids rebalance on brief disconnect [OK]

Hint: Static ID lets consumer resume without rebalance [OK]

Common Mistakes:

• Assuming every disconnect triggers rebalance
• Thinking consumer is removed permanently
• Believing new ID is required on reconnect

4.

Given this Kafka consumer config snippet, identify the error preventing static group membership:

props.put("group.id", "my-group");
props.put("group.instance.id", "consumer-1");
props.put("session.timeout.ms", "10000");
// Missing property?

medium

A. Missing group.protocol.type set to consumer

B. Missing enable.auto.commit property

C. Missing partition.assignment.strategy for static membership

D. Missing client.id property

Solution

1. Step 1: Check required properties for static membership

   Static membership requires group.protocol.type to be set to consumer explicitly in some Kafka versions.

2. Step 2: Identify missing property in config

   The snippet lacks group.protocol.type, which can prevent static membership from working properly.

3. Final Answer:

   Missing group.protocol.type set to consumer -> Option A

4. Quick Check:

   Static membership needs group.protocol.type=consumer [OK]

Hint: Set group.protocol.type=consumer for static membership [OK]

Common Mistakes:

• Assuming auto commit affects static membership
• Confusing client.id with group.instance.id
• Ignoring group.protocol.type requirement

5.

You have a Kafka consumer group with 3 consumers using static membership via group.instance.id. One consumer crashes and restarts quickly with the same group.instance.id. What is the expected behavior and why is static membership important here?

hard

A. The group pauses all consumption until manual intervention

B. The group avoids a rebalance, so processing continues smoothly with minimal delay

C. The restarted consumer must use a new group.instance.id or it will be rejected

D. The group triggers a rebalance to reassign partitions to other consumers immediately

Solution

1. Step 1: Analyze static membership effect on consumer restart

   Static membership keeps the consumer identity stable, so a quick restart does not cause a rebalance.

2. Step 2: Understand impact on group processing

   Because no rebalance occurs, partition assignments remain unchanged, allowing smooth and low-latency processing.

3. Final Answer:

   The group avoids a rebalance, so processing continues smoothly with minimal delay -> Option B

4. Quick Check:

   Static membership prevents rebalance on quick restart [OK]

Hint: Static membership keeps group stable during quick restarts [OK]

Common Mistakes:

• Expecting rebalance on every consumer restart
• Thinking new ID is required after crash
• Assuming group pauses automatically