Partitioner behavior in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the partition chosen for a given key?

Given a Kafka producer configured with 3 partitions and the default partitioner, what partition will the message with key "user123" be sent to?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer<String, String> producer = new KafkaProducer<>(props);

String topic = "test-topic";
String key = "user123";
int numPartitions = 3;

// Assume default partitioner uses key.hashCode() % numPartitions
int partition = Math.abs(key.hashCode()) % numPartitions;
System.out.println(partition);

A 1

B 2

C 0

D 3

💡 Hint

Remember the default partitioner uses the hash code of the key modulo the number of partitions.

The default Kafka partitioner calculates the partition by taking the absolute value of the key's hash code modulo the number of partitions. For key "user123" and 3 partitions, the calculation results in partition 1.

❓ Predict Output

intermediate

What happens if the key is null?

In Kafka's default partitioner, what partition will a message be sent to if the key is null and the topic has 4 partitions?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer<String, String> producer = new KafkaProducer<>(props);

String topic = "test-topic";
String key = null;
int numPartitions = 4;

// Default partitioner behavior for null key
int partition = new Random().nextInt(numPartitions);
System.out.println(partition);

A Partition 4 (out of range)

B Always partition 0

C Random partition between 0 and 3

D Throws NullPointerException

💡 Hint

Check how the default partitioner handles null keys.

When the key is null, Kafka's default partitioner assigns the message to a random partition among the available partitions.

🧠 Conceptual

advanced

Why use a custom partitioner?

Which of the following is the main reason to implement a custom partitioner in Kafka?

A To automatically retry failed messages

B To encrypt messages before sending

C To improve message serialization speed

D To control how messages are distributed across partitions based on custom logic

💡 Hint

Think about what partitioning controls in Kafka.

A custom partitioner lets you decide which partition a message goes to based on your own rules, helping with load balancing or message ordering.

❓ Predict Output

advanced

What error occurs with invalid partition number?

What error will Kafka producer throw if you try to send a message to partition 5 on a topic with only 4 partitions (0 to 3)?

Kafka

ProducerRecord<String, String> record = new ProducerRecord<>("test-topic", 5, "key", "value");
producer.send(record);

A IllegalArgumentException: Partition out of range

B NullPointerException

C No error, message sent to partition 0

D TimeoutException

💡 Hint

Check Kafka's validation for partition numbers.

Kafka producer validates the partition number and throws IllegalArgumentException if it is outside the valid range.

🧠 Conceptual

expert

How does Kafka ensure message order within a partition?

Which statement best explains how Kafka guarantees message order within a partition?

A Messages are ordered by timestamp regardless of partition

B Messages with the same key always go to the same partition, preserving order for that key

C Kafka uses a global sequence number for all messages

D Messages are randomly assigned to partitions to balance load

💡 Hint

Think about how keys and partitions relate to ordering.

Kafka guarantees order only within a partition. By sending messages with the same key to the same partition, order is preserved for that key.

Practice

1.

What does a Kafka partitioner do?

partitioner decides:

easy

A. Which partition a message is sent to

B. How to compress messages

C. The message format

D. The topic name

Solution

1. Step 1: Understand the role of a partitioner

   A partitioner decides the partition inside a Kafka topic where a message will be stored.

2. Step 2: Match the function to the options

   Only Which partition a message is sent to correctly describes this role.

3. Final Answer:

   Which partition a message is sent to -> Option A

4. Quick Check:

   Partitioner = decides message partition [OK]

Hint: Partitioner chooses partition for messages [OK]

Common Mistakes:

• Confusing partitioner with compression
• Thinking partitioner sets topic name
• Assuming partitioner changes message format

2.

Which of the following is the correct way to specify a default partitioner class in Kafka producer configuration?

props.put("partitioner.class", ?);

easy

A. "org.apache.kafka.clients.producer.KeyPartitioner"

B. "org.apache.kafka.clients.producer.RoundRobinPartitioner"

C. "org.apache.kafka.clients.producer.CustomPartitioner"

D. "org.apache.kafka.clients.producer.internals.DefaultPartitioner"

Solution

1. Step 1: Recall default partitioner class name

   The default partitioner class in Kafka is DefaultPartitioner located in org.apache.kafka.clients.producer.internals.

2. Step 2: Match the correct class name

   "org.apache.kafka.clients.producer.internals.DefaultPartitioner" matches the default partitioner class name exactly.

3. Final Answer:

   "org.apache.kafka.clients.producer.internals.DefaultPartitioner" -> Option D

4. Quick Check:

   Default partitioner class = DefaultPartitioner [OK]

Hint: Default partitioner class is DefaultPartitioner [OK]

Common Mistakes:

• Using non-existent class names
• Confusing round-robin with default partitioner
• Misspelling package or class names

3.

Given a Kafka topic with 3 partitions and a producer sending messages with keys "key1", "key2", and "key1" again, which partitions will the messages go to if the default partitioner is used?

medium

A. [0, 0, 1]

B. [1, 2, 0]

C. [0, 1, 0]

D. [2, 1, 0]

Solution

1. Step 1: Understand key-based partitioning

   Messages with the same key go to the same partition. Different keys may go to different partitions.

2. Step 2: Assign partitions for keys

   Assuming hash(key) mod 3, "key1" maps to partition 0, "key2" maps to partition 1. So messages go to partitions [0, 1, 0].

3. Final Answer:

   [0, 1, 0] -> Option C

4. Quick Check:

   Same key = same partition [OK]

Hint: Same key always same partition [OK]

Common Mistakes:

• Assuming keys distribute randomly
• Assigning different partitions to same key
• Ignoring modulo partition count

4.

Consider this code snippet for sending messages without keys:

for i in range(4):
    producer.send("topic", value=f"msg{i}")

What is the likely issue if all messages end up in the same partition?

medium

A. Messages have keys that force same partition

B. Topic has only one partition

C. The producer is not configured to use round-robin partitioning

D. Producer is using a custom partitioner ignoring keys

Solution

1. Step 1: Analyze message keys and partition count

   Messages have no keys, so default partitioner uses round-robin if multiple partitions exist.

2. Step 2: Check topic partition count

   If all messages go to the same partition, likely the topic has only one partition.

3. Final Answer:

   Topic has only one partition -> Option B

4. Quick Check:

   Single partition topic = all messages same partition [OK]

Hint: Single partition topic sends all messages there [OK]

Common Mistakes:

• Assuming round-robin works with one partition
• Thinking keys are present when they are not
• Ignoring topic partition count

5.

You want to ensure messages with the same user ID always go to the same partition, but messages without user ID keys should be evenly distributed. Which partitioner behavior should you rely on?

hard

A. Default partitioner: keyed messages to same partition, unkeyed round-robin

B. Custom partitioner that ignores keys and uses random partition

C. Round-robin partitioner for all messages

D. Partitioner that sends all messages to partition 0

Solution

1. Step 1: Understand keyed message behavior

   Default partitioner sends messages with the same key to the same partition.

2. Step 2: Understand unkeyed message behavior

   Messages without keys are distributed evenly using round-robin by default partitioner.

3. Step 3: Match behavior to requirement

   Default partitioner: keyed messages to same partition, unkeyed round-robin matches the requirement exactly.

4. Final Answer:

   Default partitioner: keyed messages to same partition, unkeyed round-robin -> Option A

5. Quick Check:

   Default partitioner fits keyed + unkeyed needs [OK]

Hint: Default partitioner handles keyed and unkeyed well [OK]

Common Mistakes:

• Choosing custom partitioner ignoring keys
• Using round-robin for keyed messages
• Sending all messages to one partition