What is Deserializer in Kafka: Explanation and Example

In Kafka, a deserializer converts byte data from Kafka messages back into usable objects or data types in your application. It is the opposite of a serializer and is essential for reading and processing messages correctly.

How It Works

Imagine you receive a letter written in a secret code. To understand it, you need to decode it first. In Kafka, messages are sent as bytes, which are like that secret code. A deserializer is the tool that decodes these bytes back into readable data your program can use.

When your application reads messages from Kafka, the deserializer takes the raw bytes and transforms them into objects like strings, numbers, or custom data types. This process ensures the data is in the right format for your application to work with.

Example

This example shows how to configure a Kafka consumer with a string deserializer to read messages as strings.

java

import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.StringDeserializer;
import java.util.Properties;
import java.util.Collections;

public class SimpleConsumer {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(ConsumerConfig.GROUP_ID_CONFIG, "group1");
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());

        KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
        consumer.subscribe(Collections.singletonList("my-topic"));

        consumer.poll(java.time.Duration.ofMillis(1000)).forEach(record -> {
            System.out.println("Received message: " + record.value());
        });

        consumer.close();
    }
}

Output

Received message: Hello Kafka
Received message: Welcome to Kafka deserialization

When to Use

Use a deserializer whenever you consume messages from Kafka topics. It is necessary to convert the raw byte data into a format your application understands. For example:

• If your messages are JSON strings, use a JSON deserializer to convert bytes into JSON objects.
• If your messages are simple text, use a string deserializer.
• For custom data formats, implement your own deserializer.

Without a proper deserializer, your application cannot read or process Kafka messages correctly.

Key Points

• A deserializer converts Kafka message bytes into usable data types.
• It is configured on the consumer side in Kafka.
• Common deserializers include StringDeserializer and JSON deserializer.
• Custom deserializers can be created for special data formats.

Key Takeaways

A deserializer converts raw Kafka message bytes into usable application data.

It is essential for reading messages correctly in Kafka consumers.

Use built-in deserializers like StringDeserializer or create custom ones for special formats.

Configure deserializers in consumer properties before reading messages.