Serialization (String, JSON, Avro) in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Kafka producer code snippet using String serialization?

Consider this Kafka producer code snippet that sends a message with String serialization for both key and value:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

Producer<String, String> producer = new KafkaProducer<>(props);
producer.send(new ProducerRecord<>("test-topic", "key1", "value1"));
producer.close();
System.out.println("Message sent");

What will this code print when run successfully?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

Producer<String, String> producer = new KafkaProducer<>(props);
producer.send(new ProducerRecord<>("test-topic", "key1", "value1"));
producer.close();
System.out.println("Message sent");

A Exception in thread "main" java.lang.ClassNotFoundException: org.apache.kafka.common.serialization.StringSerializer

B Message sent

C null

D value1

💡 Hint

Think about what the System.out.println line outputs after sending the message.

The code sends a message asynchronously and then prints "Message sent". The send method does not block or return the message content, so the only output is the print statement.

❓ Predict Output

intermediate

What is the output of this Kafka consumer code snippet using JSON deserialization?

Given this Kafka consumer code snippet that uses JSON deserialization for values:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "group1");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.connect.json.JsonDeserializer");

KafkaConsumer<String, Map<String, Object>> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Collections.singletonList("json-topic"));
ConsumerRecords<String, Map<String, Object>> records = consumer.poll(Duration.ofMillis(1000));
for (ConsumerRecord<String, Map<String, Object>> record : records) {
    System.out.println(record.value().get("name"));
}
consumer.close();

If the topic has one message with value {"name":"Alice","age":30}, what will be printed?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "group1");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.connect.json.JsonDeserializer");

KafkaConsumer<String, Map<String, Object>> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Collections.singletonList("json-topic"));
ConsumerRecords<String, Map<String, Object>> records = consumer.poll(Duration.ofMillis(1000));
for (ConsumerRecord<String, Map<String, Object>> record : records) {
    System.out.println(record.value().get("name"));
}
consumer.close();

A null

B {"name":"Alice","age":30}

C Alice

D 30

💡 Hint

Look at what key is accessed from the deserialized JSON map.

The JSON deserializer converts the JSON string into a Map. Accessing get("name") returns the string "Alice".

❓ Predict Output

advanced

What error does this Kafka producer code raise when using Avro serialization without schema registry?

Consider this Kafka producer code snippet that tries to send an Avro serialized message without configuring schema registry:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");

Producer<String, GenericRecord> producer = new KafkaProducer<>(props);
GenericRecord record = new GenericData.Record(schema);
record.put("field", "value");
producer.send(new ProducerRecord<>("avro-topic", "key1", record));
producer.close();

What error will this code most likely raise?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");

Producer<String, GenericRecord> producer = new KafkaProducer<>(props);
GenericRecord record = new GenericData.Record(schema);
record.put("field", "value");
producer.send(new ProducerRecord<>("avro-topic", "key1", record));
producer.close();

A io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: HTTP 404 Not Found

B No error, message sent successfully

C ClassNotFoundException: io.confluent.kafka.serializers.KafkaAvroSerializer

D NullPointerException

💡 Hint

Think about what happens if the schema registry URL is not set but Avro serializer is used.

The Avro serializer tries to register or fetch the schema from the schema registry. Without the URL configured, it will fail with a 404 error from the REST client.

🧠 Conceptual

advanced

Which serialization format is best suited for schema evolution in Kafka?

In Kafka, you want to choose a serialization format that supports schema evolution, allowing you to add or remove fields without breaking consumers. Which format is best suited for this?

A Plain byte array serialization

B JSON serialization

C String serialization

D Avro serialization with schema registry

💡 Hint

Consider which format uses a schema registry to manage schemas and compatibility.

Avro serialization with schema registry supports schema evolution by managing schemas centrally and ensuring compatibility between producers and consumers.

❓ Predict Output

expert

What is the number of items in the resulting Map after this JSON deserialization in Kafka consumer?

Given this Kafka consumer code snippet using JSON deserialization:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "group1");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.connect.json.JsonDeserializer");

KafkaConsumer<String, Map<String, Object>> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Collections.singletonList("json-topic"));
ConsumerRecords<String, Map<String, Object>> records = consumer.poll(Duration.ofMillis(1000));
for (ConsumerRecord<String, Map<String, Object>> record : records) {
    Map<String, Object> valueMap = record.value();
    System.out.println(valueMap.size());
}
consumer.close();

If the topic has a message with JSON value {"name":"Bob","age":25,"city":"NY"}, what will be printed?

Kafka

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "group1");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.connect.json.JsonDeserializer");

KafkaConsumer<String, Map<String, Object>> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Collections.singletonList("json-topic"));
ConsumerRecords<String, Map<String, Object>> records = consumer.poll(Duration.ofMillis(1000));
for (ConsumerRecord<String, Map<String, Object>> record : records) {
    Map<String, Object> valueMap = record.value();
    System.out.println(valueMap.size());
}
consumer.close();

A 3

B 1

C 0

D Throws NullPointerException

💡 Hint

Count the number of keys in the JSON object after deserialization.

The JSON object has three keys: "name", "age", and "city". The deserializer converts it into a Map with three entries, so size() returns 3.

Practice

1. What is the main purpose of serialization in Kafka?

easy

A. To convert data into a format suitable for transmission or storage

B. To encrypt data for security

C. To compress data to save space

D. To validate data against a schema

Solution

1. Step 1: Understand serialization concept

   Serialization means converting data into a format that can be sent or saved easily.

2. Step 2: Identify Kafka's use of serialization

   Kafka uses serialization to prepare data for sending between producers and consumers.

3. Final Answer:

   To convert data into a format suitable for transmission or storage -> Option A

4. Quick Check:

   Serialization = data format conversion [OK]

Hint: Serialization means data format conversion for sending [OK]

Common Mistakes:

• Confusing serialization with encryption
• Thinking serialization compresses data
• Assuming serialization validates data

2. Which of the following is the correct way to configure a Kafka producer to use JSON serialization in Java?

easy

A. props.put("value.serializer", "org.apache.kafka.connect.json.JsonSerializer");

B. props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

C. props.put("value.serializer", "org.apache.kafka.common.serialization.ByteArraySerializer");

D. props.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");

Solution

1. Step 1: Identify JSON serializer class

   The JSON serializer class in Kafka Connect is "org.apache.kafka.connect.json.JsonSerializer".

2. Step 2: Match configuration property

   The producer property for value serializer is "value.serializer" and should be set to the JSON serializer class.

3. Final Answer:

   props.put("value.serializer", "org.apache.kafka.connect.json.JsonSerializer"); -> Option A

4. Quick Check:

   JSON serializer = org.apache.kafka.connect.json.JsonSerializer [OK]

Hint: JSON serializer class is in kafka.connect.json package [OK]

Common Mistakes:

• Using StringSerializer for JSON data
• Confusing Avro serializer with JSON serializer
• Using ByteArraySerializer for JSON

3. Given the following Kafka producer code snippet using Avro serialization, what will be the output if the schema registry URL is incorrect?

props.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");
props.put("schema.registry.url", "http://wrong-url:8081");
producer.send(new ProducerRecord<>("topic", avroRecord));

medium

A. NullPointerException during serialization

B. Message sent successfully without errors

C. SerializationException due to schema registry connection failure

D. Producer silently drops the message

Solution

1. Step 1: Understand Avro serialization dependency

   Avro serialization requires connecting to the schema registry to fetch schemas.

2. Step 2: Analyze effect of wrong schema registry URL

   If the URL is wrong, the serializer cannot fetch the schema and throws a SerializationException.

3. Final Answer:

   SerializationException due to schema registry connection failure -> Option C

4. Quick Check:

   Wrong schema URL = SerializationException [OK]

Hint: Avro needs schema registry; wrong URL causes SerializationException [OK]

Common Mistakes:

• Expecting message to send without error
• Thinking NullPointerException occurs
• Assuming message is dropped silently

4. Identify the error in this Kafka producer configuration for String serialization:

Properties props = new Properties();
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("bootstrap.servers", "localhost:9092");
Producer producer = new KafkaProducer<>(props);

medium

A. Incorrect generic types in KafkaProducer declaration

B. Using StringDeserializer instead of StringSerializer for value.serializer

C. Missing bootstrap.servers configuration

D. No error, configuration is correct

Solution

1. Step 1: Check serializer and deserializer usage

   Producer requires serializers for key and value, not deserializers.

2. Step 2: Identify wrong class usage

   Value serializer is incorrectly set to StringDeserializer instead of StringSerializer.

3. Final Answer:

   Using StringDeserializer instead of StringSerializer for value.serializer -> Option B

4. Quick Check:

   Producer needs serializers, not deserializers [OK]

Hint: Producer uses serializers, not deserializers [OK]

Common Mistakes:

• Mixing serializer and deserializer classes
• Forgetting bootstrap.servers (not asked here)
• Incorrect generic types (not the main error)

5. You want to send complex nested data with schema evolution support in Kafka. Which serialization method should you choose and why?

hard

A. ByteArray serialization, because it handles any data type

B. JSON serialization, because it supports nested data and is human-readable

C. String serialization, because it is simple and fast

D. Avro serialization, because it supports schema evolution and compact binary format

Solution

1. Step 1: Understand requirements for nested data and schema evolution

   Complex nested data needs a format that supports schemas and changes over time.

2. Step 2: Compare serialization options

   String is simple but no schema support; JSON supports nested but no schema evolution; Avro supports both schema evolution and compact binary format.

3. Final Answer:

   Avro serialization, because it supports schema evolution and compact binary format -> Option D

4. Quick Check:

   Schema evolution + nested data = Avro [OK]

Hint: Avro supports schema evolution and nested data best [OK]

Common Mistakes:

• Choosing String for complex data
• Assuming JSON supports schema evolution
• Using ByteArray without schema control