Custom SerDes in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this custom SerDes serialization?

Given the following Kafka custom serializer code snippet, what will be the output byte array when serializing the string "abc"?

Kafka

public class CustomStringSerializer implements org.apache.kafka.common.serialization.Serializer<String> {
    @Override
    public byte[] serialize(String topic, String data) {
        if (data == null) return null;
        byte[] originalBytes = data.getBytes();
        byte[] result = new byte[originalBytes.length + 1];
        result[0] = (byte) originalBytes.length;
        System.arraycopy(originalBytes, 0, result, 1, originalBytes.length);
        return result;
    }
}

A [99, 98, 97]

B [0, 97, 98, 99]

C [3, 97, 98, 99]

D [97, 98, 99]

💡 Hint

Think about how the length of the string is stored in the first byte.

The serializer prepends the length of the string as the first byte, then copies the original bytes. For "abc", length is 3, ASCII codes are 97, 98, 99.

❓ Predict Output

intermediate

What error occurs when deserializing null data?

Consider this Kafka custom deserializer snippet. What error will occur if data is null?

Kafka

public class CustomStringDeserializer implements org.apache.kafka.common.serialization.Deserializer<String> {
    @Override
    public String deserialize(String topic, byte[] data) {
        int length = data[0];
        return new String(data, 1, length);
    }
}

A ClassCastException

B ArrayIndexOutOfBoundsException

C No error, returns empty string

D NullPointerException

💡 Hint

What happens if you try to access an element of a null array?

If data is null, accessing data[0] causes a NullPointerException.

🧠 Conceptual

advanced

Which statement best describes the role of a custom SerDes in Kafka?

Choose the most accurate description of what a custom SerDes (Serializer/Deserializer) does in Kafka.

A It manages Kafka topic partitions and offsets automatically.

B It converts data between Kafka's internal byte format and application-specific objects.

C It encrypts messages to secure Kafka communication.

D It balances load between Kafka brokers.

💡 Hint

Think about how Kafka sends and receives data as bytes but applications use objects.

Custom SerDes convert between bytes and objects so Kafka can store and transmit data while applications can work with meaningful types.

🔧 Debug

advanced

Why does this custom deserializer produce incorrect strings?

Given this deserializer code, why might the output strings be incorrect or corrupted?

Kafka

public class FaultyDeserializer implements org.apache.kafka.common.serialization.Deserializer<String> {
    @Override
    public String deserialize(String topic, byte[] data) {
        if (data == null) return null;
        int length = data[0];
        return new String(data, 0, length);
    }
}

A It includes the length byte in the string, causing corruption.

B It throws NullPointerException when data is null.

C It uses the wrong character encoding by default.

D It reverses the byte order before decoding.

💡 Hint

Check the substring start index and length used in the String constructor.

The code uses new String(data, 0, length), which includes the first byte (length byte) as a character, corrupting the output.

🚀 Application

expert

How many items are in the map after this custom SerDes deserialization?

Assume a custom deserializer converts a byte array representing multiple key-value pairs into a Map<String, Integer>. The byte array encodes 3 pairs: {"a":1, "b":2, "c":3}. After deserialization, how many entries does the map contain?

Kafka

byte[] data = new byte[]{3, 97, 1, 98, 2, 99, 3}; // Custom format: first byte count, then pairs of (keyByte, valueByte)
// Deserializer logic:
// int count = data[0];
// for i in 0..count-1:
//   key = String.valueOf((char) data[2*i+1]);
//   value = data[2*i+2];
//   map.put(key, value);

A 3

B 6

C 1

D 0

💡 Hint

Count how many key-value pairs the first byte indicates and how the loop processes them.

The first byte is 3, indicating 3 pairs. The loop runs 3 times, adding 3 entries to the map.

Practice

1.

What is the main purpose of a custom SerDes in Kafka?

easy

A. To manage Kafka topic partitions

B. To speed up Kafka brokers

C. To control how data is converted to bytes and back

D. To monitor Kafka consumer lag

Solution

1. Step 1: Understand SerDes role

   SerDes stands for serializer-deserializer, which converts data to bytes and back.

2. Step 2: Identify main purpose

   Custom SerDes lets you control this conversion for special or complex data.

3. Final Answer:

   To control how data is converted to bytes and back -> Option C

4. Quick Check:

   Custom SerDes = Data conversion control [OK]

Hint: SerDes = Serializer + Deserializer for data [OK]

Common Mistakes:

• Confusing SerDes with Kafka broker settings
• Thinking SerDes manages partitions
• Assuming SerDes monitors consumer lag

2.

Which interface must you implement to create a custom serializer in Kafka?

easy

A. Serializer<T>

B. Consumer<T>

C. Producer<T>

D. Deserializer<T>

Solution

1. Step 1: Recall serializer interface

   Kafka requires implementing Serializer<T> to convert objects to bytes.

2. Step 2: Match interface to purpose

   Deserializer<T> is for reading bytes back, not serialization.

3. Final Answer:

   Serializer<T> -> Option A

4. Quick Check:

   Serializer interface = Serializer<T> [OK]

Hint: Serializer interface handles object to bytes [OK]

Common Mistakes:

• Choosing Deserializer for serialization
• Confusing Producer/Consumer with SerDes
• Using wrong generic interface

3.

Given this serializer code snippet, what will serialize() return for input "hello"?

public byte[] serialize(String topic, String data) {
    return data == null ? null : data.getBytes(StandardCharsets.UTF_8);
}

medium

A. [104, 101, 108, 108, 111]

B. null

C. ["h", "e", "l", "l", "o"]

D. Throws NullPointerException

Solution

1. Step 1: Understand serialize method

   If data is not null, it returns UTF-8 bytes of the string.

2. Step 2: Convert "hello" to bytes

   "hello" in UTF-8 bytes is [104, 101, 108, 108, 111].

3. Final Answer:

   [104, 101, 108, 108, 111] -> Option A

4. Quick Check:

   "hello" bytes = [104,101,108,108,111] [OK]

Hint: String.getBytes(UTF-8) returns byte array [OK]

Common Mistakes:

• Thinking it returns characters array
• Assuming null is returned for non-null input
• Expecting exception on valid input

4.

Identify the error in this deserializer code snippet:

public String deserialize(String topic, byte[] data) {
    return new String(data);
}

medium

A. No error, code is correct

B. Method should return byte[] not String

C. Parameter 'topic' is unused

D. Missing charset in String constructor

Solution

1. Step 1: Check String constructor usage

   Using new String(byte[]) without charset uses platform default, which can cause bugs.

2. Step 2: Identify best practice

   Always specify charset like StandardCharsets.UTF_8 for consistent decoding.

3. Final Answer:

   Missing charset in String constructor -> Option D

4. Quick Check:

   Specify charset in String(byte[]) [OK]

Hint: Always specify charset when converting bytes to String [OK]

Common Mistakes:

• Ignoring charset causes wrong decoding
• Thinking unused parameter is an error
• Assuming return type must be byte[]

5.

You want to send a complex object Person{name, age} through Kafka. Which approach best uses custom SerDes?

1. Create a PersonSerializer that converts Person to JSON bytes.
2. Create a PersonDeserializer that converts JSON bytes back to Person.
3. Register these SerDes in Kafka producer and consumer configs.

What is the main benefit of this approach?

hard

A. It speeds up Kafka message delivery

B. It allows sending complex objects as bytes safely and readably

C. It automatically partitions topics by age

D. It removes the need for Kafka brokers

Solution

1. Step 1: Understand custom SerDes for complex objects

   Serializing to JSON bytes lets complex objects be sent as byte arrays.

2. Step 2: Recognize benefit of JSON format

   JSON is readable and widely supported, making data safe and easy to deserialize.

3. Step 3: Confirm Kafka config usage

   Registering SerDes ensures Kafka knows how to convert data on send and receive.

4. Final Answer:

   It allows sending complex objects as bytes safely and readably -> Option B

5. Quick Check:

   Custom SerDes + JSON = safe complex data transfer [OK]

Hint: Use JSON SerDes for complex object safety [OK]

Common Mistakes:

• Thinking SerDes speeds up Kafka internals
• Assuming SerDes controls topic partitions
• Believing Kafka brokers are not needed