Consumer offset commit strategies in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a consumer offset in Kafka?

A consumer offset is a marker that shows the position of a consumer in a Kafka topic partition, indicating which messages have been read.

beginner

What are the two main types of offset commit strategies in Kafka?

The two main types are automatic commit and manual commit. Automatic commits happen periodically, while manual commits are controlled by the consumer code.

intermediate

How does automatic offset commit work in Kafka consumers?

Kafka consumers automatically commit offsets at intervals defined by the configuration (e.g., every 5 seconds), which means the consumer periodically saves its position without explicit code.

intermediate

Why might you prefer manual offset commits over automatic commits?

Manual commits give you control to commit offsets only after processing messages successfully, reducing the risk of losing data or processing duplicates if the consumer crashes.

advanced

What is the difference between synchronous and asynchronous manual offset commits?

Synchronous commits block the consumer until the commit is acknowledged, ensuring the offset is saved before continuing. Asynchronous commits send the commit request and continue without waiting, which is faster but less safe.

What does an offset represent in Kafka?

A The timestamp of the last message

B The size of a Kafka topic

C The number of partitions in a topic

D The position of a consumer in a topic partition

An offset marks the position of a consumer in a topic partition, showing which messages have been read.

Which Kafka consumer setting controls automatic offset commits?

A enable.auto.commit

B auto.offset.reset

C fetch.min.bytes

D max.poll.records

The 'enable.auto.commit' setting enables or disables automatic offset commits.

What is a key advantage of manual offset commits?

A Automatic error handling

B Control over when offsets are saved

C Faster message consumption

D No need to write commit code

Manual commits let you decide exactly when to save offsets, improving reliability.

What happens if you use asynchronous offset commits?

A Commit requests are sent without waiting for confirmation

B Consumer blocks until commit is confirmed

C Offsets are never committed

D Consumer automatically retries commits

Asynchronous commits send the commit request and continue without waiting for confirmation.

If a consumer crashes before committing offsets, what risk exists with automatic commits?

A Consumer will not restart

B Messages will be lost forever

C Messages might be processed twice

D Partitions will be deleted

If offsets are committed automatically before processing completes, a crash can cause duplicate processing.

Explain the difference between automatic and manual offset commit strategies in Kafka.

Think about who controls when the offset is saved.

Describe the pros and cons of synchronous versus asynchronous manual offset commits.

Consider speed versus safety.