Partition assignment in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is partition assignment in Kafka?

Partition assignment is the process where Kafka assigns partitions of a topic to different consumer instances in a consumer group to balance load and ensure each partition is consumed by only one consumer at a time.

beginner

Name two common partition assignment strategies in Kafka.

Two common strategies are:
1. Range Assignor - assigns consecutive partitions to consumers.
2. RoundRobin Assignor - assigns partitions evenly in a round-robin fashion across consumers.

intermediate

What happens when a new consumer joins a Kafka consumer group regarding partition assignment?

Kafka triggers a rebalance where partitions are reassigned among all consumers so the load is balanced, and each partition is assigned to only one consumer.

intermediate

Explain the role of the ConsumerCoordinator in partition assignment.

The ConsumerCoordinator manages group membership and triggers partition assignment by communicating with the Kafka broker's group coordinator to assign partitions to consumers.

beginner

Why is it important that each partition is assigned to only one consumer in a group?

To avoid duplicate processing and ensure message order within a partition, Kafka guarantees that only one consumer reads from a partition at a time.

What does Kafka do when a consumer leaves a consumer group?

A It triggers a rebalance to reassign partitions.

B It deletes the topic.

C It pauses all consumers.

D It duplicates partitions.

When a consumer leaves, Kafka triggers a rebalance to redistribute partitions among remaining consumers.

Which partition assignment strategy assigns consecutive partitions to consumers?

A RoundRobin Assignor

B Random Assignor

C Sticky Assignor

D Range Assignor

Range Assignor assigns consecutive partitions to each consumer.

In Kafka, can two consumers in the same group consume the same partition simultaneously?

A No, only one consumer per partition.

B Yes, always.

C Only if configured manually.

D Only for compacted topics.

Kafka ensures only one consumer in a group reads from a partition at a time to maintain order and avoid duplicates.

What triggers a partition reassignment in Kafka consumer groups?

A Broker restart

B Topic creation

C Consumer group rebalance

D Message production

Partition reassignment happens during consumer group rebalances triggered by membership changes.

Which Kafka component coordinates partition assignment among consumers?

A Producer

B ConsumerCoordinator

C Zookeeper

D Broker Leader

The ConsumerCoordinator manages group membership and partition assignment.

Describe how Kafka assigns partitions to consumers in a consumer group and why this is important.

Think about how Kafka keeps work divided fairly and orderly among consumers.

Explain what happens during a rebalance in Kafka consumer groups and what triggers it.

Consider what changes when consumers come or go.

Practice

1.

What is the main purpose of partition assignment in Kafka?

easy

A. To encrypt messages before sending

B. To split topic data among consumers to share the workload

C. To compress data for faster transmission

D. To create new topics automatically

Solution

1. Step 1: Understand partition assignment role

   Partition assignment divides topic partitions among consumers to distribute processing.

2. Step 2: Compare options

   Only To split topic data among consumers to share the workload describes splitting data among consumers; others describe unrelated tasks.

3. Final Answer:

   To split topic data among consumers to share the workload -> Option B

4. Quick Check:

   Partition assignment = workload sharing [OK]

Hint: Partition assignment shares topic data among consumers [OK]

Common Mistakes:

• Confusing partition assignment with encryption
• Thinking it compresses data
• Assuming it creates topics

2.

Which of the following is a valid Kafka partition assignment strategy?

easy

A. RoundRobin

B. HashMap

C. QuickSort

D. BinarySearch

Solution

1. Step 1: Recall Kafka partition assignment strategies

   Kafka supports strategies like Range, RoundRobin, and Sticky.

2. Step 2: Identify valid strategy

   RoundRobin is a known Kafka strategy; others are unrelated algorithms.

3. Final Answer:

   RoundRobin -> Option A

4. Quick Check:

   RoundRobin = valid strategy [OK]

Hint: Remember Kafka uses RoundRobin, Range, Sticky strategies [OK]

Common Mistakes:

• Confusing general algorithms with Kafka strategies
• Choosing unrelated computer science terms
• Ignoring Kafka-specific terms

3.

Given a topic with 4 partitions and 2 consumers using the Range strategy, how are partitions assigned?

Partitions: 0, 1, 2, 3
Consumers: C1, C2
Strategy: Range

medium

A. Partitions are assigned randomly

B. C1 gets partitions 0 and 2; C2 gets partitions 1 and 3

C. C1 gets all partitions; C2 gets none

D. C1 gets partitions 0 and 1; C2 gets partitions 2 and 3

Solution

1. Step 1: Understand Range strategy behavior

   Range assigns consecutive partitions to each consumer in order.

2. Step 2: Apply Range to 4 partitions and 2 consumers

   Partitions 0 and 1 go to C1; partitions 2 and 3 go to C2.

3. Final Answer:

   C1 gets partitions 0 and 1; C2 gets partitions 2 and 3 -> Option D

4. Quick Check:

   Range = consecutive partition blocks [OK]

Hint: Range assigns consecutive partitions to consumers [OK]

Common Mistakes:

• Mixing RoundRobin with Range strategy
• Assuming random assignment
• Giving all partitions to one consumer

4.

Identify the error in this partition assignment code snippet using the Sticky strategy:

assignor = StickyAssignor()
assignor.assign(partitions=[0,1,2], consumers=["C1"])

medium

A. Consumers list must be integers

B. Partitions list should be empty

C. StickyAssignor requires at least two consumers for assignment

D. StickyAssignor does not support partition assignment

Solution

1. Step 1: Understand StickyAssignor requirements

   Sticky strategy balances partitions among multiple consumers to minimize movement.

2. Step 2: Check given consumers list

   Only one consumer is provided, so StickyAssignor cannot balance partitions.

3. Final Answer:

   StickyAssignor requires at least two consumers for assignment -> Option C

4. Quick Check:

   Sticky needs multiple consumers [OK]

Hint: Sticky strategy needs multiple consumers to balance partitions [OK]

Common Mistakes:

• Assuming single consumer works with Sticky
• Confusing consumer data types
• Thinking Sticky doesn't assign partitions

5.

You have 5 consumers and a topic with 10 partitions. You want to maintain processing order per partition and balance load evenly. Which partition assignment strategy should you choose?

hard

A. Sticky, because it balances partitions while minimizing reassignment

B. RoundRobin, because it distributes partitions evenly across consumers

C. Range, because it assigns consecutive partitions to consumers

D. Random, because it assigns partitions unpredictably

Solution

1. Step 1: Analyze requirements

   Need to keep processing order per partition and balance load evenly.

2. Step 2: Compare strategies

   Range assigns consecutive partitions but may imbalance load; RoundRobin balances evenly but can cause frequent partition moves; Sticky balances load and minimizes partition reassignment, preserving order.

3. Step 3: Choose best fit

   Sticky strategy best meets both order and load balance requirements.

4. Final Answer:

   Sticky, because it balances partitions while minimizing reassignment -> Option A

5. Quick Check:

   Sticky = balanced load + order preservation [OK]

Hint: Sticky balances load and keeps partition order [OK]

Common Mistakes:

• Choosing Range despite load imbalance
• Picking RoundRobin ignoring partition order
• Selecting random assignment which is unpredictable