Schema evolution (backward, forward, full) in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to set the schema compatibility to backward in Kafka.

Kafka

props.put("schema.compatibility.level", "[1]");

Drag options to blanks, or click blank then click option'

Abackward

Bforward

Cfull

Dnone

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'forward' instead of 'backward' for backward compatibility.

Setting schema.compatibility.level to backward ensures new schemas can read data produced with previous schemas.

2fill in blank

medium

Complete the code to set the schema compatibility to forward in Kafka.

Kafka

props.put("schema.compatibility.level", "[1]");

Drag options to blanks, or click blank then click option'

Afull

Bnone

Cbackward

Dforward

Attempts:

3 left

💡 Hint

Common Mistakes

Confusing forward with backward compatibility.

Setting schema.compatibility.level to forward ensures old schemas can read data produced with new schemas.

3fill in blank

hard

Fix the error in setting the schema compatibility to full in Kafka.

Kafka

props.put("schema.compatibility.level", "[1]");

Drag options to blanks, or click blank then click option'

Abackward

Bfull

Cforward

Dnone

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'backward' or 'forward' instead of 'full' for full compatibility.

Full compatibility means schemas are both backward and forward compatible.

4fill in blank

hard

Fill both blanks to create a schema registry configuration with backward compatibility and a URL.

Kafka

props.put("schema.compatibility.level", "[1]");
props.put("schema.registry.url", "[2]");

Drag options to blanks, or click blank then click option'

Abackward

Bhttp://localhost:8081

Cforward

Dfull

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing forward and backward compatibility or wrong URL format.

The first blank sets backward compatibility, the second sets the schema registry URL.

5fill in blank

hard

Fill all three blanks to configure full compatibility, schema registry URL, and subject name strategy.

Kafka

props.put("schema.compatibility.level", "[1]");
props.put("schema.registry.url", "[2]");
props.put("value.subject.name.strategy", "[3]");

Drag options to blanks, or click blank then click option'

Abackward

Bhttp://schema-registry:8081

Cio.confluent.kafka.serializers.subject.TopicNameStrategy

Dfull

Attempts:

3 left

💡 Hint

Common Mistakes

Using wrong compatibility level or incorrect subject name strategy class.

Full compatibility is set, the schema registry URL points to the service, and the subject name strategy is set to TopicNameStrategy.

Practice

1. What does backward compatibility mean in Kafka schema evolution?

easy

A. New consumers can read data produced with the old schema.

B. Old consumers can read data produced with the new schema.

C. Both old and new consumers can read all data versions.

D. Schemas cannot be changed once set.

Solution

1. Step 1: Understand backward compatibility meaning

   Backward compatibility means older consumers can still read data written with a newer schema.

2. Step 2: Match definition to options

   Old consumers can read data produced with the new schema. states old consumers can read new data, which matches backward compatibility.

3. Final Answer:

   Old consumers can read data produced with the new schema. -> Option B

4. Quick Check:

   Backward compatibility = old consumers read new data [OK]

Hint: Backward means old consumers read new data [OK]

Common Mistakes:

• Confusing backward with forward compatibility
• Thinking backward means new consumers read old data
• Assuming schemas cannot change

2. Which of the following is the correct way to declare a new optional field with a default value in an Avro schema for forward compatibility?

easy

A. {"name": "age", "type": "string", "default": "unknown"}

B. {"name": "age", "type": "int", "default": null}

C. {"name": "age", "type": "int", "default": 0}

D. {"name": "age", "type": "int"}

Solution

1. Step 1: Identify forward compatibility rule

   Forward compatibility requires new fields to have default values so new consumers can read data produced with the old schema.

2. Step 2: Check options for default value and type

   {"name": "age", "type": "int", "default": 0} declares an int field with default 0, which is correct for forward compatibility.

3. Final Answer:

   {"name": "age", "type": "int", "default": 0} -> Option C

4. Quick Check:

   New optional field with default = {"name": "age", "type": "int", "default": 0} [OK]

Hint: New fields need defaults for forward compatibility [OK]

Common Mistakes:

• Missing default value for new fields
• Using null default without union type
• Setting wrong type for default

3. Given this Avro schema evolution scenario:

Old schema: {"name": "id", "type": "int"}
New schema: {"name": "id", "type": "int"}, {"name": "name", "type": "string", "default": "unknown"}

What compatibility type does this change support?

medium

A. Backward compatible only

B. Forward compatible only

C. Not compatible

D. Full compatible

Solution

1. Step 1: Analyze schema change

   A new field "name" with a default value "unknown" is added to the schema.

2. Step 2: Determine compatibility

   Adding a field with a default supports both backward and forward compatibility, so it is full compatible.

3. Final Answer:

   Full compatible -> Option D

4. Quick Check:

   Adding field with default = full compatibility [OK]

Hint: Adding defaulted fields means full compatibility [OK]

Common Mistakes:

• Thinking adding fields breaks backward compatibility
• Ignoring default values
• Confusing forward and backward compatibility

4. You have this Avro schema change:

Old schema: {"name": "email", "type": "string"}
New schema: {"name": "email", "type": ["null", "string"], "default": null}

What is the main issue preventing backward compatibility?

medium

A. Changing field type from string to union breaks old consumers.

B. Adding default null makes schema incompatible.

C. Removing the field causes backward incompatibility.

D. No issue; schema is backward compatible.

Solution

1. Step 1: Identify schema change impact

   The field type changed from string to a union of null and string, which is a type change.

2. Step 2: Understand backward compatibility rules

   Changing a field's type breaks backward compatibility because old consumers expect a string, not a union.

3. Final Answer:

   Changing field type from string to union breaks old consumers. -> Option A

4. Quick Check:

   Type change breaks backward compatibility [OK]

Hint: Changing field type breaks backward compatibility [OK]

Common Mistakes:

• Assuming default null causes incompatibility
• Thinking adding union is always safe
• Ignoring type changes impact

5. You want to evolve a Kafka Avro schema to add a new field status that must be backward and forward compatible. Which approach is best?

hard

A. Add status as an optional string field with default value "active".

B. Add status as a required string field without default.

C. Remove an old optional field and add status as required.

D. Change an existing field type to string and add status without default.

Solution

1. Step 1: Understand compatibility requirements

   Backward and forward compatibility require new fields to have default values and be optional.

2. Step 2: Evaluate options

   Add status as an optional string field with default value "active". adds status as optional with default "active", satisfying both compatibilities.

3. Final Answer:

   Add status as an optional string field with default value "active". -> Option A

4. Quick Check:

   Optional field with default ensures full compatibility [OK]

Hint: New fields need defaults for full compatibility [OK]

Common Mistakes:

• Adding required fields without defaults
• Removing fields breaks compatibility
• Changing existing field types