Producer throughput optimization in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to set the batch size for the Kafka producer.

Kafka

props.put("batch.size", [1]);

Drag options to blanks, or click blank then click option'

A16384

B1024

C512

D2048

Attempts:

3 left

💡 Hint

Common Mistakes

Using too small batch size reduces throughput.

Setting batch size to zero disables batching.

The batch.size property controls the batch size in bytes. Setting it to 16384 (16 KB) is a common default for better throughput.

2fill in blank

medium

Complete the code to set linger.ms to delay sending messages for batching.

Kafka

props.put("linger.ms", [1]);

Drag options to blanks, or click blank then click option'

A100

B5

C0

D1000

Attempts:

3 left

💡 Hint

Common Mistakes

Setting linger.ms to 0 disables batching delay.

Setting linger.ms too high increases latency.

Setting linger.ms to 5 milliseconds allows the producer to wait briefly to batch more messages, improving throughput.

3fill in blank

hard

Fix the error in setting compression.type to improve throughput.

Kafka

props.put("compression.type", [1]);

Drag options to blanks, or click blank then click option'

A"gzip"

Bgzip

C"none"

D"snappy"

Attempts:

3 left

💡 Hint

Common Mistakes

Omitting quotes causes a syntax error.

Using unsupported compression types.

The compression.type property expects a string value. Using "gzip" enables gzip compression, which can improve throughput by reducing message size.

4fill in blank

hard

Fill both blanks to create a producer config that sets max.in.flight.requests.per.connection and enables idempotence.

Kafka

props.put("max.in.flight.requests.per.connection", [1]);
props.put("enable.idempotence", [2]);

Drag options to blanks, or click blank then click option'

A5

B"true"

C"false"

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Using boolean true instead of string "true" for idempotence.

Setting max.in.flight.requests.per.connection too high.

Setting max.in.flight.requests.per.connection to 5 allows multiple requests in flight, and enable.idempotence to "true" ensures exactly-once delivery.

5fill in blank

hard

Fill all three blanks to configure retries, acks, and request.timeout.ms for better throughput and reliability.

Kafka

props.put("retries", [1]);
props.put("acks", [2]);
props.put("request.timeout.ms", [3]);

Drag options to blanks, or click blank then click option'

A3

B"all"

C30000

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Using acks value without quotes.

Setting retries to 0 disables retries.

Setting retries to 3 allows retrying failed sends, acks to "all" waits for all replicas, and request.timeout.ms to 30000 ms sets a 30-second timeout for requests.

Practice

1. What is the main benefit of increasing the batch.size setting in a Kafka producer?

easy

A. It guarantees message delivery to all replicas.

B. It compresses messages to save disk space.

C. It allows sending more messages together, improving throughput.

D. It reduces the number of partitions in a topic.

Solution

1. Step 1: Understand batch.size role

   The batch.size controls how many messages the producer groups before sending.

2. Step 2: Effect on throughput

   Grouping more messages reduces network calls, increasing throughput.

3. Final Answer:

   It allows sending more messages together, improving throughput. -> Option C

4. Quick Check:

   batch.size increases throughput [OK]

Hint: More batch size means fewer sends, better throughput [OK]

Common Mistakes:

• Confusing batch.size with message compression
• Thinking batch.size controls message durability
• Assuming batch.size changes topic partitions

2. Which of the following is the correct way to set the linger time to 100 milliseconds in Kafka producer configuration?

easy

A. "linger.ms = 100"

B. "linger_time = 100"

C. "lingerMs: 100"

D. "linger = 100ms"

Solution

1. Step 1: Identify correct property name

   The Kafka producer uses linger.ms to set linger time in milliseconds.

2. Step 2: Confirm correct syntax

   The correct syntax is linger.ms = 100, using dot notation and milliseconds.

3. Final Answer:

   "linger.ms = 100" -> Option A

4. Quick Check:

   linger.ms sets linger time [OK]

Hint: Use exact property names with dots, like linger.ms [OK]

Common Mistakes:

• Using underscores instead of dots
• Adding units like 'ms' in the value
• Using camelCase or colon syntax incorrectly

3. Given this Kafka producer config snippet:

props.put("batch.size", 16384);
props.put("linger.ms", 5);
props.put("acks", "all");

What is the expected effect on message sending?

medium

A. Messages batch up to 16KB or wait 5ms before sending, with full acknowledgment.

B. Messages are compressed automatically before sending.

C. Messages are sent immediately without batching.

D. Producer waits for only one replica acknowledgment.

Solution

1. Step 1: Analyze batch.size and linger.ms

   Batch size 16384 bytes means messages batch up to 16KB or linger 5ms before sending.

2. Step 2: Understand acks setting

   acks=all means producer waits for all replicas to confirm, ensuring durability.

3. Final Answer:

   Messages batch up to 16KB or wait 5ms before sending, with full acknowledgment. -> Option A

4. Quick Check:

   batch.size + linger.ms + acks=all = Messages batch up to 16KB or wait 5ms before sending, with full acknowledgment. [OK]

Hint: Batch size + linger.ms controls send timing, acks controls durability [OK]

Common Mistakes:

• Assuming messages send immediately ignoring linger.ms
• Confusing acks=all with acks=1
• Thinking compression happens automatically without config

4. You have this Kafka producer config:

props.put("batch.size", 32768);
props.put("linger.ms", 0);
props.put("acks", "0");

What is the main problem with this setup if you want high throughput and reliability?

medium

A. linger.ms=0 disables batching, reducing throughput.

B. acks=0 means no acknowledgment, risking message loss.

C. batch.size is too large and causes errors.

D. The config syntax is invalid and will cause exceptions.

Solution

1. Step 1: Check acks setting impact

   acks=0 means producer does not wait for any acknowledgment, risking message loss.

2. Step 2: Evaluate throughput vs reliability

   While batch.size is large and linger.ms=0 disables waiting, the main reliability issue is acks=0.

3. Final Answer:

   acks=0 means no acknowledgment, risking message loss. -> Option B

4. Quick Check:

   acks=0 risks loss despite throughput [OK]

Hint: acks=0 disables reliability, not good for safe throughput [OK]

Common Mistakes:

• Thinking linger.ms=0 disables batching completely
• Assuming batch.size too large causes errors
• Confusing config syntax as invalid

5. You want to optimize a Kafka producer for maximum throughput but still ensure messages are not lost. Which combination of settings is best?

hard

A. batch.size=512, linger.ms=50, acks=0, compression.type=lz4

B. batch.size=1024, linger.ms=0, acks=0, compression.type=none

C. batch.size=16384, linger.ms=0, acks=1, compression.type=gzip

D. batch.size=65536, linger.ms=20, acks=all, compression.type=snappy

Solution

1. Step 1: Maximize throughput with batching and linger

   Large batch.size (65536) and linger.ms (20ms) allow grouping many messages before sending.

2. Step 2: Ensure reliability and compression

   acks=all ensures all replicas confirm, preventing loss. Compression (snappy) reduces data size, improving throughput.

3. Final Answer:

   batch.size=65536, linger.ms=20, acks=all, compression.type=snappy -> Option D

4. Quick Check:

   Large batch + linger + acks=all + compression = batch.size=65536, linger.ms=20, acks=all, compression.type=snappy [OK]

Hint: Big batch + linger + acks=all + compression = best throughput + safety [OK]

Common Mistakes:

• Choosing acks=0 losing reliability
• Using too small batch sizes
• Ignoring compression benefits