Batch size and compression tuning in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is batch size in Kafka producer configuration?

Batch size is the maximum amount of data (in bytes) the producer will collect before sending it to the broker. Larger batch sizes can improve throughput by sending more data at once.

beginner

How does compression affect Kafka producer performance?

Compression reduces the size of data sent over the network, which can improve throughput and reduce bandwidth usage. However, it requires CPU resources to compress and decompress data.

beginner

Name three common compression types supported by Kafka producers.

Kafka supports these compression types:

• none (no compression)
• gzip
• snappy
• lz4
• zstd

intermediate

What is the trade-off when increasing batch size in Kafka producers?

Increasing batch size improves throughput by sending more data at once but can increase latency because the producer waits longer to fill the batch before sending.

intermediate

Why might you choose lz4 compression over gzip in Kafka?

lz4 compresses and decompresses faster than gzip, which reduces CPU usage and latency, though gzip usually achieves better compression ratios (smaller data size).

What does increasing the batch size in Kafka producer typically improve?

A Throughput

B Latency

C Security

D Data format

Increasing batch size allows sending more data at once, improving throughput.

Which Kafka compression type is known for fastest compression and decompression?

A none

B snappy

C gzip

D lz4

lz4 is designed for very fast compression and decompression.

What is a downside of using compression in Kafka producers?

A Increased network bandwidth

B Higher CPU usage

C Lower throughput

D Data loss

Compression requires CPU resources to compress and decompress data.

If you want to reduce latency in Kafka producer, what should you do with batch size?

A Enable compression

B Increase batch size

C Decrease batch size

D Disable retries

Smaller batch sizes reduce wait time before sending, lowering latency.

Which compression type generally achieves the smallest data size in Kafka?

A gzip

B lz4

C snappy

D none

gzip usually compresses data more tightly than other types.

Explain how batch size affects Kafka producer performance and latency.

Think about how waiting to fill a batch changes speed and delay.

Describe the benefits and costs of enabling compression in Kafka producers.

Consider both network and CPU effects.