Serialization (String, JSON, Avro) in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to serialize a string message before sending it to Kafka.

Kafka

producer.send(topic, value=[1].encode('utf-8'))

Drag options to blanks, or click blank then click option'

Astr(message)

Bjson.dumps(message)

Cmessage

Dbytes(message)

Attempts:

3 left

💡 Hint

Common Mistakes

Trying to send the message without encoding it.

Using json.dumps on a plain string.

To send a string message to Kafka, you encode it to bytes using encode('utf-8').

2fill in blank

medium

Complete the code to serialize a Python dictionary to JSON string before sending.

Kafka

producer.send(topic, value=[1].encode('utf-8'))

Drag options to blanks, or click blank then click option'

Ajson.dumps(data)

Bdata

Cstr(data)

Dpickle.dumps(data)

Attempts:

3 left

💡 Hint

Common Mistakes

Using str() which does not produce valid JSON.

Sending the dict directly without serialization.

Use json.dumps() to convert a dictionary to a JSON string before encoding.

3fill in blank

hard

Fix the error in deserializing a JSON message received from Kafka.

Kafka

data = json.loads(message.[1]('utf-8'))

Drag options to blanks, or click blank then click option'

Aencode

Bto_bytes

Cto_str

Ddecode

Attempts:

3 left

💡 Hint

Common Mistakes

Using encode instead of decode causes errors.

Trying to load JSON directly from bytes.

Kafka messages are bytes, so decode to string before loading JSON.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that serializes only keys with length greater than 3 to JSON strings.

Kafka

result = {key: json.dumps(value) for key, value in data.items() if len(key) [1] 3 and key [2] 'id'}

Drag options to blanks, or click blank then click option'

A>

B<

C!=

D==

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<' instead of '>' for length comparison.

Using '==' instead of '!=' to exclude keys.

We want keys longer than 3 and keys not equal to 'id'.

5fill in blank

hard

Fill all three blanks to create a dictionary comprehension that serializes keys in uppercase, values only if they are integers, and filters values greater than 10.

Kafka

result = [1]: [2] for [3], v in data.items() if isinstance(v, int) and v > 10}

Drag options to blanks, or click blank then click option'

Ak.upper()

Bv

Ck

Djson.dumps(v)

Attempts:

3 left

💡 Hint

Common Mistakes

Using v directly without serialization.

Using wrong variable names for keys.

Keys are uppercased with k.upper(), values are serialized with json.dumps(v), and k is the variable for keys.

Practice

1. What is the main purpose of serialization in Kafka?

easy

A. To convert data into a format suitable for transmission or storage

B. To encrypt data for security

C. To compress data to save space

D. To validate data against a schema

Solution

1. Step 1: Understand serialization concept

   Serialization means converting data into a format that can be sent or saved easily.

2. Step 2: Identify Kafka's use of serialization

   Kafka uses serialization to prepare data for sending between producers and consumers.

3. Final Answer:

   To convert data into a format suitable for transmission or storage -> Option A

4. Quick Check:

   Serialization = data format conversion [OK]

Hint: Serialization means data format conversion for sending [OK]

Common Mistakes:

• Confusing serialization with encryption
• Thinking serialization compresses data
• Assuming serialization validates data

2. Which of the following is the correct way to configure a Kafka producer to use JSON serialization in Java?

easy

A. props.put("value.serializer", "org.apache.kafka.connect.json.JsonSerializer");

B. props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

C. props.put("value.serializer", "org.apache.kafka.common.serialization.ByteArraySerializer");

D. props.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");

Solution

1. Step 1: Identify JSON serializer class

   The JSON serializer class in Kafka Connect is "org.apache.kafka.connect.json.JsonSerializer".

2. Step 2: Match configuration property

   The producer property for value serializer is "value.serializer" and should be set to the JSON serializer class.

3. Final Answer:

   props.put("value.serializer", "org.apache.kafka.connect.json.JsonSerializer"); -> Option A

4. Quick Check:

   JSON serializer = org.apache.kafka.connect.json.JsonSerializer [OK]

Hint: JSON serializer class is in kafka.connect.json package [OK]

Common Mistakes:

• Using StringSerializer for JSON data
• Confusing Avro serializer with JSON serializer
• Using ByteArraySerializer for JSON

3. Given the following Kafka producer code snippet using Avro serialization, what will be the output if the schema registry URL is incorrect?

props.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroSerializer");
props.put("schema.registry.url", "http://wrong-url:8081");
producer.send(new ProducerRecord<>("topic", avroRecord));

medium

A. NullPointerException during serialization

B. Message sent successfully without errors

C. SerializationException due to schema registry connection failure

D. Producer silently drops the message

Solution

1. Step 1: Understand Avro serialization dependency

   Avro serialization requires connecting to the schema registry to fetch schemas.

2. Step 2: Analyze effect of wrong schema registry URL

   If the URL is wrong, the serializer cannot fetch the schema and throws a SerializationException.

3. Final Answer:

   SerializationException due to schema registry connection failure -> Option C

4. Quick Check:

   Wrong schema URL = SerializationException [OK]

Hint: Avro needs schema registry; wrong URL causes SerializationException [OK]

Common Mistakes:

• Expecting message to send without error
• Thinking NullPointerException occurs
• Assuming message is dropped silently

4. Identify the error in this Kafka producer configuration for String serialization:

Properties props = new Properties();
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("bootstrap.servers", "localhost:9092");
Producer producer = new KafkaProducer<>(props);

medium

A. Incorrect generic types in KafkaProducer declaration

B. Using StringDeserializer instead of StringSerializer for value.serializer

C. Missing bootstrap.servers configuration

D. No error, configuration is correct

Solution

1. Step 1: Check serializer and deserializer usage

   Producer requires serializers for key and value, not deserializers.

2. Step 2: Identify wrong class usage

   Value serializer is incorrectly set to StringDeserializer instead of StringSerializer.

3. Final Answer:

   Using StringDeserializer instead of StringSerializer for value.serializer -> Option B

4. Quick Check:

   Producer needs serializers, not deserializers [OK]

Hint: Producer uses serializers, not deserializers [OK]

Common Mistakes:

• Mixing serializer and deserializer classes
• Forgetting bootstrap.servers (not asked here)
• Incorrect generic types (not the main error)

5. You want to send complex nested data with schema evolution support in Kafka. Which serialization method should you choose and why?

hard

A. ByteArray serialization, because it handles any data type

B. JSON serialization, because it supports nested data and is human-readable

C. String serialization, because it is simple and fast

D. Avro serialization, because it supports schema evolution and compact binary format

Solution

1. Step 1: Understand requirements for nested data and schema evolution

   Complex nested data needs a format that supports schemas and changes over time.

2. Step 2: Compare serialization options

   String is simple but no schema support; JSON supports nested but no schema evolution; Avro supports both schema evolution and compact binary format.

3. Final Answer:

   Avro serialization, because it supports schema evolution and compact binary format -> Option D

4. Quick Check:

   Schema evolution + nested data = Avro [OK]

Hint: Avro supports schema evolution and nested data best [OK]

Common Mistakes:

• Choosing String for complex data
• Assuming JSON supports schema evolution
• Using ByteArray without schema control