Cooperative vs eager rebalancing in Kafka - Practice Questions

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of consumer group rebalance with eager strategy

Consider a Kafka consumer group with two consumers consuming from four partitions. The group uses eager rebalancing. What is the expected behavior during a rebalance when a new consumer joins?

Kafka

Consumer group with 2 consumers consuming 4 partitions.
New consumer joins triggering eager rebalance.
What happens to partition assignment during rebalance?

A All consumers stop consuming, partitions are revoked from all, then reassigned to all consumers.

B Only the new consumer starts consuming immediately, others continue without interruption.

C Partitions are reassigned incrementally without stopping any consumer.

D Consumers continue consuming old partitions until manually restarted.

💡 Hint

Think about how eager rebalancing handles partition revocation and assignment.

Eager rebalancing revokes all partitions from all consumers before reassigning them. This causes a full stop and restart of consumption during rebalance.

❓ Predict Output

intermediate

Output behavior with cooperative rebalancing

In a Kafka consumer group using cooperative rebalancing, what happens when a new consumer joins the group?

Kafka

Consumer group with 3 consumers consuming 6 partitions.
New consumer joins triggering cooperative rebalance.
What is the behavior during partition reassignment?

A Partitions are revoked and reassigned incrementally, allowing consumers to continue processing some partitions without interruption.

B All partitions are revoked from all consumers before reassignment, causing a full stop.

C New consumer cannot join until all consumers restart manually.

D Partitions are duplicated temporarily to all consumers during rebalance.

💡 Hint

Cooperative rebalancing tries to minimize disruption. How does it do that?

Cooperative rebalancing revokes partitions incrementally, so consumers keep processing some partitions while others are reassigned, reducing downtime.

🧠 Conceptual

advanced

Why choose cooperative over eager rebalancing?

Which of the following is the main advantage of cooperative rebalancing compared to eager rebalancing in Kafka consumer groups?

A It guarantees faster rebalancing by revoking all partitions at once.

B It requires fewer network resources by disabling partition reassignment.

C It allows consumers to consume duplicate partitions temporarily.

D It reduces consumer downtime by allowing incremental partition revocation and assignment.

💡 Hint

Think about how downtime affects consumer processing.

Cooperative rebalancing reduces downtime by revoking partitions incrementally, so consumers keep processing some partitions during rebalance.

❓ Predict Output

advanced

Error raised when using cooperative rebalancing with unsupported client

What error will a Kafka consumer raise if it tries to use cooperative rebalancing but the client library does not support it?

Kafka

KafkaConsumer consumer = new KafkaConsumer(props);
consumer.subscribe(topics, new CooperativeRebalanceListener());
consumer.poll(Duration.ofMillis(100));

A java.lang.NullPointerException

B org.apache.kafka.common.errors.UnsupportedVersionException

C org.apache.kafka.common.errors.TimeoutException

D No error, cooperative rebalancing silently falls back to eager

💡 Hint

Check what happens when a feature is not supported by the Kafka broker or client.

If the client or broker does not support cooperative rebalancing, an UnsupportedVersionException is thrown indicating the feature is unavailable.

🚀 Application

expert

Designing a consumer group for minimal downtime during rebalance

You are designing a Kafka consumer group for a critical application that must minimize downtime during rebalances caused by scaling consumers up or down. Which rebalance strategy and configuration should you choose?

A Use eager rebalancing but increase session timeout to delay rebalances.

B Use eager rebalancing with default full partition revocation and assignment.

C Use cooperative rebalancing with incremental partition revocation and assignment enabled.

D Disable rebalancing and manually assign partitions to consumers.

💡 Hint

Consider which strategy reduces consumer downtime and supports dynamic scaling.

Cooperative rebalancing allows incremental partition revocation and assignment, minimizing downtime during scaling events, making it ideal for critical applications.

Practice

1. What is the main advantage of cooperative rebalancing in Kafka consumer groups?

easy

A. It reduces consumer pause time by reassigning partitions gradually.

B. It reassigns all partitions at once for faster processing.

C. It eliminates the need for consumer group coordination.

D. It increases the number of partitions automatically.

Solution

1. Step 1: Understand cooperative rebalancing behavior

   Cooperative rebalancing allows consumers to leave partitions gradually, which reduces the pause time during rebalancing.

2. Step 2: Compare with eager rebalancing

   Eager rebalancing reassigns all partitions at once, causing longer pauses, unlike cooperative rebalancing.

3. Final Answer:

   It reduces consumer pause time by reassigning partitions gradually. -> Option A

4. Quick Check:

   Cooperative rebalancing = gradual partition reassignment [OK]

Hint: Cooperative means gradual, eager means all at once [OK]

Common Mistakes:

• Confusing eager with cooperative rebalancing
• Thinking cooperative rebalancing eliminates coordination
• Assuming rebalancing increases partitions

2. Which configuration property enables cooperative rebalancing in Kafka consumers?

easy

A. group.rebalance.strategy = 'eager'

B. rebalance.policy = 'cooperative'

C. enable.cooperative.rebalance = true

D. partition.assignment.strategy = 'org.apache.kafka.clients.consumer.CooperativeStickyAssignor'

Solution

1. Step 1: Identify the correct property for assignment strategy

   Kafka uses partition.assignment.strategy to set the rebalance strategy class.

2. Step 2: Recognize the cooperative assignor class

   The cooperative rebalancing strategy is implemented by org.apache.kafka.clients.consumer.CooperativeStickyAssignor.

3. Final Answer:

   partition.assignment.strategy = 'org.apache.kafka.clients.consumer.CooperativeStickyAssignor' -> Option D

4. Quick Check:

   Cooperative assignor class = partition.assignment.strategy = 'org.apache.kafka.clients.consumer.CooperativeStickyAssignor' [OK]

Hint: Look for 'partition.assignment.strategy' with cooperative class [OK]

Common Mistakes:

• Using incorrect property names
• Confusing eager and cooperative strategy names
• Assuming a boolean flag enables cooperative rebalancing

3. Given two consumers in a group using eager rebalancing, what happens when one consumer leaves?

medium

A. All partitions in the group are reassigned immediately, causing a pause.

B. Only the partitions of the leaving consumer are reassigned gradually.

C. The remaining consumer continues without any rebalancing.

D. Partitions are reassigned only after a timeout period.

Solution

1. Step 1: Understand eager rebalancing behavior on consumer leave

   In eager rebalancing, when any consumer leaves, all partitions are reassigned immediately to maintain balance.

2. Step 2: Recognize the impact on consumer pause

   This immediate reassignment causes a pause for all consumers during the rebalance.

3. Final Answer:

   All partitions in the group are reassigned immediately, causing a pause. -> Option A

4. Quick Check:

   Eager rebalancing = full immediate reassignment [OK]

Hint: Eager means all partitions reassign at once [OK]

Common Mistakes:

• Thinking only leaving consumer's partitions reassign
• Assuming no pause occurs
• Believing reassignment waits for timeout

4. You have a Kafka consumer group using cooperative rebalancing but notice long pauses during rebalances. What is a likely cause?

medium

A. Partitions are being reassigned gradually as expected.

B. The consumer group has only one consumer.

C. Consumers are using the eager assignor instead of cooperative assignor.

D. The rebalance timeout is set too low.

Solution

1. Step 1: Check the assignor configuration

   If long pauses occur despite expecting cooperative rebalancing, the consumers might be using the eager assignor by mistake.

2. Step 2: Understand the impact of eager assignor

   Eager assignor causes all partitions to reassign at once, leading to longer pauses.

3. Final Answer:

   Consumers are using the eager assignor instead of cooperative assignor. -> Option C

4. Quick Check:

   Long pauses + cooperative expected = eager assignor used [OK]

Hint: Long pauses mean eager assignor, not cooperative [OK]

Common Mistakes:

• Assuming gradual reassignment causes long pauses
• Ignoring assignor configuration
• Blaming rebalance timeout without checking assignor

5. You want to minimize downtime in a Kafka consumer group with frequent consumer joins and leaves. Which strategy and configuration should you choose?

hard

A. Use eager rebalancing with default assignor for simplicity.

B. Use cooperative rebalancing with CooperativeStickyAssignor to reduce pause time.

C. Disable rebalancing to avoid any pauses.

D. Use eager rebalancing but increase session timeout to reduce rebalances.

Solution

1. Step 1: Identify the goal

   The goal is to minimize downtime during frequent consumer group changes.

2. Step 2: Choose the appropriate rebalance strategy

   Cooperative rebalancing reduces pause time by reassigning partitions gradually, which suits frequent joins/leaves.

3. Step 3: Select the correct configuration

   Using CooperativeStickyAssignor enables cooperative rebalancing in Kafka consumers.

4. Final Answer:

   Use cooperative rebalancing with CooperativeStickyAssignor to reduce pause time. -> Option B

5. Quick Check:

   Minimize downtime = cooperative rebalancing + correct assignor [OK]

Hint: CooperativeStickyAssignor minimizes downtime with frequent changes [OK]

Common Mistakes:

• Choosing eager rebalancing for low downtime
• Disabling rebalancing causing stale assignments
• Misconfiguring session timeout instead of strategy