Producer retries and idempotency in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to enable idempotence for the Kafka producer.

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("[1]", "true");

Drag options to blanks, or click blank then click option'

Aacks

Benable.idempotence

Cretries

Dmax.in.flight.requests.per.connection

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'acks' instead of 'enable.idempotence'.

Setting 'retries' to true (it's a number, not boolean).

Setting enable.idempotence to true enables idempotent producer behavior in Kafka.

2fill in blank

medium

Complete the code to set the number of retries for the Kafka producer.

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("retries", "[1]");

Drag options to blanks, or click blank then click option'

A3

B-1

C0

Dtrue

Attempts:

3 left

💡 Hint

Common Mistakes

Using boolean values like 'true' for retries.

Setting retries to '-1' which means infinite retries but is less common.

The retries property expects a string representing a number. Setting it to "3" means the producer will retry sending messages up to 3 times.

3fill in blank

hard

Fix the error in the code to ensure safe retries with idempotency.

Kafka

props.put("max.in.flight.requests.per.connection", "[1]");

Drag options to blanks, or click blank then click option'

A0

B10

C5

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Using higher values like 5 or 10 which can cause message reordering.

Setting it to 0 which is invalid.

Setting max.in.flight.requests.per.connection to "1" ensures that only one request is sent at a time, preventing message reordering during retries when idempotence is enabled.

4fill in blank

hard

Fill both blanks to configure the producer for idempotent retries with proper acknowledgments.

Kafka

props.put("[1]", "all");
props.put("[2]", "true");

Drag options to blanks, or click blank then click option'

Aacks

Benable.idempotence

Cretries

Dmax.in.flight.requests.per.connection

Attempts:

3 left

💡 Hint

Common Mistakes

Confusing 'retries' with 'acks'.

Not enabling idempotence while setting acks to 'all'.

Setting acks to "all" ensures the leader and all replicas acknowledge the message. Enabling enable.idempotence to "true" ensures safe retries without duplicates.

5fill in blank

hard

Fill all three blanks to create a Kafka producer configuration that safely retries with idempotency and limits in-flight requests.

Kafka

props.put("[1]", "true");
props.put("[2]", "3");
props.put("[3]", "1");

Drag options to blanks, or click blank then click option'

Aenable.idempotence

Bretries

Cmax.in.flight.requests.per.connection

Dacks

Attempts:

3 left

💡 Hint

Common Mistakes

Setting 'acks' instead of 'max.in.flight.requests.per.connection' for the last property.

Not enabling idempotence while setting retries.

This configuration enables idempotency, sets retries to 3, and limits in-flight requests to 1 to prevent message reordering during retries.

Practice

1. What is the main purpose of enabling retries in a Kafka producer?

easy

A. To resend messages automatically if sending fails

B. To speed up message delivery

C. To prevent duplicate messages

D. To encrypt messages before sending

Solution

1. Step 1: Understand retries in Kafka producer

   Retries allow the producer to resend messages if the first attempt fails due to temporary issues.

2. Step 2: Identify the main purpose

   The main goal is to improve message delivery success by retrying failed sends.

3. Final Answer:

   To resend messages automatically if sending fails -> Option A

4. Quick Check:

   Retries = Resend on failure [OK]

Hint: Retries mean automatic resend on failure [OK]

Common Mistakes:

• Confusing retries with idempotency
• Thinking retries speed up delivery
• Assuming retries encrypt messages

2. Which configuration must be set to true to enable idempotency in Kafka producer?

easy

A. retries = 0

B. enable.idempotency = false

C. enable.idempotency = true

D. acks = 0

Solution

1. Step 1: Identify idempotency configuration

   The Kafka producer uses enable.idempotency to control idempotency behavior.

2. Step 2: Determine correct setting

   Setting enable.idempotency to true activates idempotency to avoid duplicates.

3. Final Answer:

   enable.idempotency = true -> Option C

4. Quick Check:

   Idempotency enabled = true [OK]

Hint: Idempotency config is enable.idempotency = true [OK]

Common Mistakes:

• Setting enable.idempotency to false
• Confusing retries with idempotency config
• Using acks = 0 disables guarantees

3. Given the following Kafka producer settings:

enable.idempotency = true
retries = 3
acks = 'all'

What is the expected behavior if a message send fails temporarily?

medium

A. The producer will stop sending messages

B. The message will be sent only once, no retries

C. Duplicates may occur due to retries

D. The message will be retried up to 3 times without duplicates

Solution

1. Step 1: Analyze retries and idempotency settings

   Retries = 3 means the producer will retry sending up to 3 times on failure. Idempotency = true prevents duplicates.

2. Step 2: Understand combined effect

   Retries with idempotency ensure retries do not cause duplicate messages.

3. Final Answer:

   The message will be retried up to 3 times without duplicates -> Option D

4. Quick Check:

   Retries + idempotency = safe retries [OK]

Hint: Retries with idempotency avoid duplicates [OK]

Common Mistakes:

• Assuming retries cause duplicates always
• Ignoring idempotency effect
• Thinking producer stops after failure

4. You have a Kafka producer with retries=5 but enable.idempotency=false. What problem might occur?

medium

A. Messages will be encrypted automatically

B. Duplicate messages may be produced on retries

C. Producer will crash on first failure

D. Messages will never be sent

Solution

1. Step 1: Understand retries without idempotency

   Retries resend messages on failure, but without idempotency duplicates can happen.

2. Step 2: Identify the problem

   Without idempotency, each retry may produce the same message again, causing duplicates.

3. Final Answer:

   Duplicate messages may be produced on retries -> Option B

4. Quick Check:

   No idempotency + retries = duplicates risk [OK]

Hint: Retries without idempotency cause duplicates [OK]

Common Mistakes:

• Thinking producer crashes on failure
• Assuming messages never send
• Confusing encryption with idempotency

5. You want to ensure exactly-once message delivery in Kafka with retries enabled. Which combination of settings should you use?

hard

A. enable.idempotency=true, retries>0, acks='all'

B. enable.idempotency=false, retries=0, acks=1

C. enable.idempotency=true, retries=0, acks=0

D. enable.idempotency=false, retries>0, acks=0

Solution

1. Step 1: Identify settings for exactly-once delivery

   Exactly-once requires idempotency enabled, retries to resend on failure, and acks='all' for full commit guarantee.

2. Step 2: Evaluate each option

   enable.idempotency=true, retries>0, acks='all' has all required settings. Others disable idempotency or retries or use weak acks.

3. Final Answer:

   enable.idempotency=true, retries>0, acks='all' -> Option A

4. Quick Check:

   Idempotency + retries + acks='all' = exactly-once [OK]

Hint: Enable idempotency, retries, and acks='all' for exactly-once [OK]

Common Mistakes:

• Disabling idempotency with retries
• Using acks=0 or 1 for strong guarantees
• Setting retries=0 disables resend