Static group membership in Kafka - Step-by-Step Execution

Process Flow - Static group membership

Start Consumer

↓

Join Group Request

↓

Static Member ID Provided?

No→Assign Dynamic Member ID

|Yes↓

Join Group with Static Member ID

↓

Group Coordinator Accepts

↓

Consumer Joins Group

↓

Consumer Maintains Session

↓

Rebalance Triggered?

No→Continue Processing

Yes↓

Static Member Rejoins with Same ID

↓

Group Stabilizes Without Losing Member

↓

Continue Processing

The consumer joins a group using a static member ID to keep membership stable across restarts, avoiding unnecessary rebalances.

Execution Sample

Kafka

consumerProps.put(ConsumerConfig.GROUP_ID_CONFIG, "my-group");
consumerProps.put(ConsumerConfig.GROUP_INSTANCE_ID_CONFIG, "consumer-1");
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(consumerProps);
consumer.subscribe(Collections.singletonList("my-topic"));
consumer.poll(Duration.ofMillis(100));

This code creates a Kafka consumer with a static group instance ID to join a consumer group with static membership.

Process Table

Step	Action	Static Member ID Provided?	Group Coordinator Response	Result
1	Consumer starts and sends JoinGroup request	Yes (consumer-1)	Accepts static member	Consumer joins group with static ID
2	Consumer polls messages	Yes (consumer-1)	Maintains session	Consumer processes messages
3	Consumer restarts and re-joins group	Yes (consumer-1)	Recognizes static member	No rebalance triggered, session continues
4	Another consumer joins with dynamic ID	No	Triggers rebalance	Group rebalances, static member remains stable
5	Static member leaves group	Yes (consumer-1)	Removes member	Group rebalances to exclude static member

💡 Execution stops when consumer leaves group or application ends.

Status Tracker

Variable	Start	After Step 1	After Step 3	After Step 4	Final
memberId	null	consumer-1	consumer-1	consumer-1	null
groupState	empty	joined	joined	rebalancing	left
rebalanceNeeded	false	false	false	true	false

Key Moments - 3 Insights

Why does the group not rebalance when the static member restarts?

Because the static member rejoins with the same member ID (see execution_table step 3), the coordinator keeps the member stable and avoids rebalance.

What happens if a consumer joins without a static member ID?

The group coordinator treats it as a dynamic member, which can trigger a rebalance (see execution_table step 4).

Why is the memberId variable null at the start and final steps?

At start, the consumer has not joined yet, so memberId is null; at final, after leaving the group, memberId resets to null (see variable_tracker).

Visual Quiz - 3 Questions

Look at the execution_table at step 3, what does the group coordinator do when the static member rejoins?

A Triggers a rebalance

B Rejects the member

C Recognizes static member and avoids rebalance

D Assigns a new dynamic member ID

💡 Hint

Check the 'Group Coordinator Response' column at step 3 in execution_table

According to variable_tracker, what is the value of 'rebalanceNeeded' after step 4?

A false

B true

C null

D undefined

💡 Hint

Look at the 'rebalanceNeeded' row and the 'After Step 4' column in variable_tracker

If the static member ID was not provided at step 1, what would likely happen?

A Consumer joins as dynamic member, possibly triggering rebalance

B Group coordinator rejects the join

C Consumer joins with static ID anyway

D Consumer cannot join the group

💡 Hint

Refer to execution_table step 4 where a consumer without static ID triggers rebalance

Concept Snapshot

Static group membership in Kafka:
- Use GROUP_INSTANCE_ID_CONFIG to set static member ID
- Static members keep same ID across restarts
- Avoids unnecessary group rebalances
- Coordinator recognizes static IDs to maintain stability
- Helps improve consumer group stability and performance

Full Transcript

Static group membership in Kafka allows consumers to join a group with a fixed member ID. This ID is set using the GROUP_INSTANCE_ID_CONFIG property. When a consumer restarts, it uses the same static ID to rejoin the group. The group coordinator recognizes this and avoids triggering a rebalance, keeping the group stable. If a consumer joins without a static ID, it is treated as dynamic and may cause rebalances. Variables like memberId track the consumer's identity, and groupState tracks the membership status. This feature improves consumer group stability by reducing unnecessary rebalances.

Practice

1.

What is the main purpose of using group.instance.id in Kafka consumer groups?

easy

A. To increase the number of partitions in a topic

B. To encrypt messages within the consumer group

C. To automatically commit offsets more frequently

D. To assign a stable identity to a consumer and enable static group membership

Solution

1. Step 1: Understand the role of group.instance.id

   This configuration assigns a unique and stable identity to a Kafka consumer within a group.

2. Step 2: Connect stable identity to static group membership

   With a stable identity, Kafka can avoid unnecessary rebalances when a consumer restarts briefly, improving stability.

3. Final Answer:

   To assign a stable identity to a consumer and enable static group membership -> Option D

4. Quick Check:

   Stable consumer identity = static group membership [OK]

Hint: Static membership means stable consumer ID to avoid rebalances [OK]

Common Mistakes:

• Confusing group.instance.id with partition count
• Thinking it controls offset commit frequency
• Assuming it encrypts messages

2.

Which of the following is the correct way to set group.instance.id in a Kafka consumer configuration?

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
// What is the correct line to set static membership?

easy

A. props.put("group.instance.id", "consumer-1");

B. props.put("group.id.instance", "consumer-1");

C. props.put("static.group.id", "consumer-1");

D. props.put("group.instance", "consumer-1");

Solution

1. Step 1: Recall the exact property name for static membership

   The correct property to set a static consumer ID is group.instance.id.

2. Step 2: Match the correct syntax in Java Properties

   Using props.put("group.instance.id", "consumer-1") correctly sets the static ID.

3. Final Answer:

   props.put("group.instance.id", "consumer-1"); -> Option A

4. Quick Check:

   Correct property name = group.instance.id [OK]

Hint: Remember exact property name: group.instance.id [OK]

Common Mistakes:

• Swapping words in the property name
• Using incomplete or incorrect property keys
• Confusing with group.id

3.

Consider this Kafka consumer group with static membership enabled using group.instance.id. If a consumer with group.instance.id = "consumer-1" disconnects briefly and reconnects, what happens?

medium

A. The consumer is removed permanently from the group

B. The group triggers a full rebalance and reassigns partitions

C. The consumer resumes without triggering a rebalance

D. The consumer must use a new group.instance.id to reconnect

Solution

1. Step 1: Understand static membership behavior on brief disconnect

   Static membership keeps the consumer identity stable, so brief disconnects do not cause rebalances.

2. Step 2: Identify the effect on partition assignment

   The consumer resumes with the same partitions without triggering a rebalance, improving latency and stability.

3. Final Answer:

   The consumer resumes without triggering a rebalance -> Option C

4. Quick Check:

   Static membership avoids rebalance on brief disconnect [OK]

Hint: Static ID lets consumer resume without rebalance [OK]

Common Mistakes:

• Assuming every disconnect triggers rebalance
• Thinking consumer is removed permanently
• Believing new ID is required on reconnect

4.

Given this Kafka consumer config snippet, identify the error preventing static group membership:

props.put("group.id", "my-group");
props.put("group.instance.id", "consumer-1");
props.put("session.timeout.ms", "10000");
// Missing property?

medium

A. Missing group.protocol.type set to consumer

B. Missing enable.auto.commit property

C. Missing partition.assignment.strategy for static membership

D. Missing client.id property

Solution

1. Step 1: Check required properties for static membership

   Static membership requires group.protocol.type to be set to consumer explicitly in some Kafka versions.

2. Step 2: Identify missing property in config

   The snippet lacks group.protocol.type, which can prevent static membership from working properly.

3. Final Answer:

   Missing group.protocol.type set to consumer -> Option A

4. Quick Check:

   Static membership needs group.protocol.type=consumer [OK]

Hint: Set group.protocol.type=consumer for static membership [OK]

Common Mistakes:

• Assuming auto commit affects static membership
• Confusing client.id with group.instance.id
• Ignoring group.protocol.type requirement

5.

You have a Kafka consumer group with 3 consumers using static membership via group.instance.id. One consumer crashes and restarts quickly with the same group.instance.id. What is the expected behavior and why is static membership important here?

hard

A. The group pauses all consumption until manual intervention

B. The group avoids a rebalance, so processing continues smoothly with minimal delay

C. The restarted consumer must use a new group.instance.id or it will be rejected

D. The group triggers a rebalance to reassign partitions to other consumers immediately

Solution

1. Step 1: Analyze static membership effect on consumer restart

   Static membership keeps the consumer identity stable, so a quick restart does not cause a rebalance.

2. Step 2: Understand impact on group processing

   Because no rebalance occurs, partition assignments remain unchanged, allowing smooth and low-latency processing.

3. Final Answer:

   The group avoids a rebalance, so processing continues smoothly with minimal delay -> Option B

4. Quick Check:

   Static membership prevents rebalance on quick restart [OK]

Hint: Static membership keeps group stable during quick restarts [OK]

Common Mistakes:

• Expecting rebalance on every consumer restart
• Thinking new ID is required after crash
• Assuming group pauses automatically