Schema evolution (backward, forward, full) in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output when a consumer with an older schema reads a message with a newer schema?

Consider a Kafka topic using Avro serialization with schema evolution enabled. The producer adds a new optional field to the schema. The consumer uses the older schema without this new field. What happens when the consumer reads the message?

Kafka

Producer schema v2: {"type":"record","name":"User","fields":[{"name":"name","type":"string"},{"name":"age","type":["null","int"],"default":null}]}
Consumer schema v1: {"type":"record","name":"User","fields":[{"name":"name","type":"string"}]}

A The consumer reads the message but sets 'age' to zero by default.

B The consumer throws a serialization error due to the missing 'age' field.

C The consumer blocks and waits for a schema update.

D The consumer reads the message successfully, ignoring the new 'age' field.

💡 Hint

Think about forward compatibility: can an older consumer read data produced with the newer schema?

Forward compatibility means new schemas can add optional fields with defaults. Older consumers ignore these new fields and read data successfully.

❓ Predict Output

intermediate

What happens when a producer sends data with an older schema and the consumer expects a newer schema?

A Kafka consumer expects a schema with a new required field added (no default). The producer sends data using the older schema without this field. What is the result when the consumer tries to deserialize the message?

Kafka

Producer schema v1: {"type":"record","name":"User","fields":[{"name":"name","type":"string"}]}
Consumer schema v2: {"type":"record","name":"User","fields":[{"name":"name","type":"string"},{"name":"email","type":"string"}]}

A The consumer throws a deserialization error due to the missing 'email' field.

B The consumer sets 'email' to an empty string by default.

C The consumer ignores the missing 'email' field and reads the message.

D The consumer reads the message but sets 'email' to null.

💡 Hint

Consider backward compatibility and required fields without defaults.

Backward compatibility requires that new schemas can read old data. But if a new required field has no default, the consumer cannot fill it, causing a deserialization error.

🧠 Conceptual

advanced

Which schema evolution mode allows both backward and forward compatibility?

In Kafka schema registry, which compatibility mode ensures that schemas are compatible both backward and forward?

A Full compatibility

B Forward compatibility

C No compatibility

D Backward compatibility

💡 Hint

Think about the mode that combines both backward and forward compatibility.

Full compatibility mode ensures that new schemas can read data produced by old schemas (backward) and old schemas can read data produced by new schemas (forward).

❓ Predict Output

advanced

What error occurs if a required field is removed in a new schema version?

A Kafka topic uses Avro schemas. The original schema has fields 'name' and 'age' (both required). The new schema removes the 'age' field. What happens when the new schema is registered with full compatibility enabled?

Kafka

Original schema: {"type":"record","name":"User","fields":[{"name":"name","type":"string"},{"name":"age","type":"int"}]}
New schema: {"type":"record","name":"User","fields":[{"name":"name","type":"string"}]}

A Schema registry accepts the new schema but warns about data loss.

B Schema registry accepts the new schema without errors.

C Schema registry rejects the new schema due to incompatibility.

D Schema registry accepts the new schema only if 'age' has a default.

💡 Hint

Removing required fields breaks forward compatibility.

Removing a required field breaks forward compatibility because old consumers expect the 'age' field but new data produced with the new schema won't include it (and old schema has no default for it). Full compatibility requires backward and forward compatibility, so the registry rejects the new schema.

🔧 Debug

expert

Why does this schema registration fail under backward compatibility?

Given the original schema and a new schema, the new schema registration fails under backward compatibility. Identify the cause.

Kafka

Original schema: {"type":"record","name":"User","fields":[{"name":"name","type":"string"},{"name":"age","type":["null","int"],"default":null}]}
New schema: {"type":"record","name":"User","fields":[{"name":"name","type":"string"},{"name":"age","type":"int"}]}

A The new schema adds a new required field without a default.

B The new schema removes the default for 'age', breaking backward compatibility.

C The new schema changes the type of 'name' field.

D The new schema adds an optional field with a default.

💡 Hint

Check changes in field defaults and types.

Backward compatibility requires that new schemas can read old data. Removing the default from a nullable field makes the new schema incompatible with old data that may have nulls.