Configuration best practices in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Kafka Producer Configuration Output

What will be the output of the following Kafka producer configuration code snippet when sending a message?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("acks", "all");
props.put("retries", 0);
props.put("batch.size", 16384);
props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

KafkaProducer<String, String> producer = new KafkaProducer<>(props);
ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key1", "value1");
producer.send(record, (metadata, exception) -> {
    if (exception == null) {
        System.out.println("Message sent to partition " + metadata.partition() + " with offset " + metadata.offset());
    } else {
        System.out.println("Error sending message: " + exception.getMessage());
    }
});
producer.close();

A No output because producer.send is asynchronous and program exits immediately

B Error sending message: UnknownTopicOrPartitionException

C Message sent to partition -1 with offset -1

D Message sent to partition 0 with offset 0

💡 Hint

Consider the default partitioning behavior and that the callback prints success or error.

The producer is configured correctly and sends a message to the topic 'my-topic'. Since no custom partitioner is set, the message goes to partition 0. The callback prints the partition and offset on success.

❓ Predict Output

intermediate

Kafka Consumer Configuration Behavior

Given this Kafka consumer configuration, what will be the value of the consumer's group.id after creation?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("enable.auto.commit", "true");
props.put("auto.commit.interval.ms", "1000");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
// Note: group.id is NOT set here

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
String groupId = consumer.groupMetadata().groupId();
System.out.println(groupId);

A Throws IllegalStateException

B default

C empty string

D null

💡 Hint

group.id is required for consumers to join a group.

Kafka consumer requires a group.id to be set. If it is missing, the consumer will throw an IllegalStateException when trying to access group metadata.

🧠 Conceptual

advanced

Best Practice for Kafka Broker Configuration

Which of the following is the best practice for configuring Kafka broker's log retention to balance disk usage and data availability?

A Set log.retention.hours to a reasonable time based on business needs and monitor disk usage

B Set log.retention.bytes to a small value to delete logs quickly

C Disable log retention and manually delete logs when disk is full

D Set log.retention.hours to a very high value and never delete old logs

💡 Hint

Think about balancing data availability and disk space.

Setting log retention to a reasonable time based on business needs ensures data is available for consumers while preventing disk from filling up. Monitoring disk usage helps adjust settings proactively.

❓ Predict Output

advanced

Effect of Incorrect Serializer Configuration

What will happen if a Kafka producer is configured with a StringSerializer for the key but an IntegerSerializer for the value, but the value sent is a String?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.IntegerSerializer");

KafkaProducer<String, String> producer = new KafkaProducer<>(props);
ProducerRecord<String, String> record = new ProducerRecord<>("my-topic", "key1", "value1");
producer.send(record);
producer.close();

A Message is sent successfully with value serialized as string

B Message is sent but value is null

C SerializationException is thrown at runtime

D Compile-time error due to type mismatch

💡 Hint

Check if the serializer matches the data type sent.

The IntegerSerializer expects Integer values. Sending a String causes a SerializationException at runtime.

🧠 Conceptual

expert

Optimizing Kafka Consumer Throughput

Which configuration change is most effective to increase Kafka consumer throughput when processing large volumes of messages?

A Set session.timeout.ms to a very low value to detect failures faster

B Set max.poll.records to a higher number and increase fetch.min.bytes

C Set enable.auto.commit to false and commit offsets manually after each message

D Set auto.offset.reset to earliest to always read from beginning

💡 Hint

Think about batch size and fetch size for throughput.

Increasing max.poll.records allows the consumer to fetch more messages per poll. Increasing fetch.min.bytes makes the broker wait to accumulate more data before responding. Both improve throughput.