Batching and linger configuration in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Effect of linger.ms on message batching

Consider a Kafka producer configured with linger.ms=100. The producer sends messages every 50ms. What is the expected behavior regarding message batching?

Kafka

producer_config = {
  'linger.ms': 100
}

# Messages sent every 50ms
# How does linger.ms affect batching here?

A Messages are sent immediately without batching because 50ms < linger.ms

B Messages are delayed indefinitely until batch size is reached

C Messages are batched and sent every 100ms or when batch is full

D Messages are sent every 50ms ignoring linger.ms

💡 Hint

Think about how linger.ms controls the maximum wait time before sending a batch.

The linger.ms setting specifies the maximum time to wait before sending a batch. Even if messages arrive earlier, the producer waits up to 100ms to batch more messages. Since messages come every 50ms, they accumulate and are sent every 100ms or when the batch is full.

❓ Predict Output

intermediate

Batch size effect on Kafka producer throughput

A Kafka producer has batch.size=16384 bytes and linger.ms=0. If the producer sends messages smaller than 1KB each rapidly, what is the expected behavior?

Kafka

producer_config = {
  'batch.size': 16384,  # 16 KB
  'linger.ms': 0
}

# Messages are ~1KB each, sent rapidly

A Messages are dropped if batch size is not reached

B Messages are batched up to 16KB before sending

C Messages are delayed up to linger.ms before sending

D Each message is sent immediately without batching

💡 Hint

What does linger.ms=0 imply about waiting for batch completion?

With linger.ms=0, the producer sends messages immediately without waiting to fill the batch. Even if batch.size is large, messages are sent as soon as they are ready.

🔧 Debug

advanced

Identify the cause of delayed message sending

A Kafka producer is configured with linger.ms=500 and batch.size=32768. However, messages are delayed longer than 500ms before sending. What is the most likely cause?

Kafka

producer_config = {
  'linger.ms': 500,
  'batch.size': 32768
}

# Messages are sent slowly, delays > 500ms observed

A The producer is waiting for acknowledgments before sending next batch

B The batch size is too large, causing delays waiting for batch to fill

C linger.ms is ignored if batch.size is not reached

D The network latency causes linger.ms to be ineffective

💡 Hint

Consider how batch.size and linger.ms interact to trigger sending.

The producer waits up to linger.ms before sending a batch, but if the batch size is large and messages arrive slowly, the batch may not fill quickly. This can cause delays longer than linger.ms if the producer is configured to wait for batch filling.

📝 Syntax

advanced

Correct Kafka producer configuration syntax for batching

Which of the following Kafka producer configuration snippets correctly sets batch.size to 32KB and linger.ms to 100ms in Java?

A

props.put("batch.size", 32 * 1024);
props.put("linger.ms", 100);

B

props.put("batch.size", "32768");
props.put("linger.ms", "100");

C

props.put(batch.size, 32768);
props.put(linger.ms, 100);

D

props.put("batch.size", 32768);
props.put("linger.ms", 100);

💡 Hint

Check for correct key names as strings and valid value types.

In Java Kafka producer configs, keys must be strings. Values can be integers or strings. Option A uses a valid expression for 32KB and correct string keys.

🚀 Application

expert

Optimizing Kafka producer for low latency and high throughput

You want to configure a Kafka producer to achieve low latency but still benefit from batching to improve throughput. Which configuration is best?

A Set linger.ms to a small positive value (e.g., 5ms) and batch.size to a moderate value

B Set linger.ms to 0 and batch.size to a large value

C Set linger.ms to a large value (e.g., 500ms) and batch.size to a small value

D Set linger.ms to a large value and batch.size to a large value

💡 Hint

Think about balancing wait time and batch size for latency and throughput.

Setting linger.ms to a small positive value allows the producer to wait briefly to batch messages, improving throughput without adding much latency. A moderate batch size complements this by not requiring too many messages to send a batch.