Consumer offset commit strategies in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Kafka consumer offset commit code?

Consider a Kafka consumer configured with enable.auto.commit=false. After processing a batch of messages, the consumer commits offsets synchronously using commitSync(). What will be the output of the following code snippet?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "test-group");
props.put("enable.auto.commit", "false");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Arrays.asList("test-topic"));

try {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
    for (ConsumerRecord<String, String> record : records) {
        System.out.println("Received: " + record.value());
    }
    consumer.commitSync();
    System.out.println("Offsets committed synchronously.");
} finally {
    consumer.close();
}

A Prints all received messages but throws an exception at commitSync()

B Prints all received messages and then prints 'Offsets committed synchronously.'

C Prints nothing and immediately prints 'Offsets committed synchronously.'

D Prints all received messages and commits offsets asynchronously without confirmation

💡 Hint

Think about what commitSync() does and how enable.auto.commit=false affects offset commits.

With enable.auto.commit=false, offsets are not committed automatically. The commitSync() method commits offsets synchronously, so after processing messages, the offsets are committed and the confirmation message is printed.

❓ Predict Output

intermediate

What happens when using commitAsync() in Kafka consumer?

Given a Kafka consumer with enable.auto.commit=false, what is the expected behavior of the following code snippet that uses commitAsync()?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "test-group");
props.put("enable.auto.commit", "false");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Arrays.asList("test-topic"));

try {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
    for (ConsumerRecord<String, String> record : records) {
        System.out.println("Received: " + record.value());
    }
    consumer.commitAsync();
    System.out.println("Offsets committed asynchronously.");
} finally {
    consumer.close();
}

A Prints all received messages and then prints 'Offsets committed asynchronously.' immediately without waiting for commit confirmation

B Prints all received messages and waits for commit confirmation before printing the message

C Throws a runtime exception because commitAsync() requires a callback

D Does not commit offsets and prints only received messages

💡 Hint

Remember that commitAsync() commits offsets in the background and does not block the thread.

The commitAsync() method commits offsets asynchronously, so the program continues immediately after calling it and prints the confirmation message without waiting for the commit to complete.

🧠 Conceptual

advanced

Which commit strategy ensures exactly-once processing in Kafka consumers?

In Kafka consumer offset commit strategies, which approach best supports exactly-once processing semantics?

A Manually committing offsets synchronously after processing each record

B Using <code>enable.auto.commit=true</code> with default auto commit interval

C Committing offsets asynchronously without handling commit failures

D Not committing offsets at all and relying on consumer restart

💡 Hint

Think about when offsets should be committed to avoid message loss or duplication.

Manually committing offsets synchronously after processing each record ensures that offsets are only committed after successful processing, supporting exactly-once semantics.

🔧 Debug

advanced

Identify the error in this Kafka consumer offset commit code

What error will this Kafka consumer code produce when trying to commit offsets?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "test-group");
props.put("enable.auto.commit", "true");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Arrays.asList("test-topic"));

ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
for (ConsumerRecord<String, String> record : records) {
    System.out.println("Received: " + record.value());
}
consumer.commitSync();

A Throws CommitFailedException because commitSync() is called with auto commit enabled

B Throws IllegalStateException because offsets are auto-committed and manual commitSync() is called

C No error; offsets are committed synchronously successfully

D Throws IllegalArgumentException due to missing commit callback

💡 Hint

Check if calling commitSync() is allowed when enable.auto.commit=true.

Even if enable.auto.commit=true, calling commitSync() is allowed and will commit offsets synchronously. No error is thrown.

🚀 Application

expert

How many offsets are committed after this batch processing with manual commit?

A Kafka consumer processes messages from partitions 0 and 1 of a topic. It processes 3 messages from partition 0 with offsets 5, 6, 7 and 2 messages from partition 1 with offsets 10, 11. After processing, it commits offsets manually using commitSync() with the following map:

{new TopicPartition("topic", 0): new OffsetAndMetadata(8), new TopicPartition("topic", 1): new OffsetAndMetadata(12)}

How many offsets are committed in total?

A 3 offsets committed, only for partition 0 messages

B 5 offsets committed, one for each message processed

C 0 offsets committed because offsets are off by one

D 2 offsets committed, one for each partition

💡 Hint

Remember that Kafka commits the offset of the next message to read, not the last processed message.

The commit map contains one offset per partition indicating the next offset to read. So two offsets are committed, one for each partition.