Schema Registry concept in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a Schema Registry in Kafka?

A Schema Registry is a service that stores and manages schemas for Kafka messages. It helps ensure that producers and consumers agree on the data format.

intermediate

Why is schema evolution important in Kafka?

Schema evolution allows schemas to change over time without breaking existing data or applications. The Schema Registry supports this by validating compatibility between schema versions.

beginner

What formats does Schema Registry commonly support?

Schema Registry commonly supports Avro, JSON Schema, and Protobuf formats for defining message schemas.

intermediate

How does Schema Registry help prevent data corruption?

By enforcing schema compatibility and validating messages against registered schemas, Schema Registry prevents producers from sending incompatible data that could corrupt consumers.

intermediate

What is the role of schema ID in Kafka messages when using Schema Registry?

The schema ID is a unique identifier stored in the message payload that tells consumers which schema version to use to decode the message.

What does Schema Registry primarily manage in Kafka?

A Kafka topics

B Message schemas

C Consumer groups

D Broker configurations

Schema Registry manages message schemas to ensure data format consistency.

Which of these is NOT a schema format supported by Schema Registry?

A Avro

B JSON Schema

C XML Schema

D Protobuf

Schema Registry does not natively support XML Schema.

What does schema compatibility ensure?

A Schemas can change without breaking consumers

B Messages are encrypted

C Topics are balanced

D Consumers read faster

Compatibility ensures schema changes do not break existing consumers.

Where is the schema ID stored in Kafka messages when using Schema Registry?

A In the broker config

B In the topic name

C In the consumer group

D In the message payload

The schema ID is embedded in the message payload to identify the schema version.

How does Schema Registry help producers and consumers?

A By enforcing agreed data formats

B By managing Kafka partitions

C By controlling network traffic

D By storing message payloads

Schema Registry enforces data format agreements between producers and consumers.

Explain what a Schema Registry is and why it is useful in Kafka.

Think about how producers and consumers agree on message structure.

Describe how schema evolution works with Schema Registry and why compatibility matters.

Consider what happens when data formats change over time.

Practice

1. What is the main purpose of a Schema Registry in Kafka?

easy

A. To encrypt Kafka messages during transmission

B. To monitor Kafka broker performance

C. To route messages between Kafka topics

D. To store and manage data schemas for Kafka messages

Solution

1. Step 1: Understand the role of Schema Registry

   Schema Registry stores and manages the structure (schema) of data used in Kafka messages.

2. Step 2: Compare with other Kafka components

   Monitoring, routing, and encryption are handled by other Kafka tools, not Schema Registry.

3. Final Answer:

   To store and manage data schemas for Kafka messages -> Option D

4. Quick Check:

   Schema Registry = data schema storage [OK]

Hint: Schema Registry manages data formats, not Kafka internals [OK]

Common Mistakes:

• Confusing Schema Registry with Kafka brokers
• Thinking it handles message routing
• Assuming it encrypts messages

2. Which of the following is the correct way to register a schema using Kafka Schema Registry API?

easy

A. DELETE /topics/{topic}/schemas with schema in YAML

B. GET /schemas/{id} with schema in XML

C. POST /subjects/{subject}/versions with schema in JSON

D. PUT /brokers/{brokerId}/schemas with schema in CSV

Solution

1. Step 1: Identify the correct HTTP method and endpoint

   Registering a schema uses POST on /subjects/{subject}/versions with JSON schema.

2. Step 2: Check other options for correctness

   GET retrieves schemas, DELETE removes topics (not schemas), PUT on brokers is invalid for schema registration.

3. Final Answer:

   POST /subjects/{subject}/versions with schema in JSON -> Option C

4. Quick Check:

   Register schema = POST with JSON [OK]

Hint: Register schemas with POST and JSON format [OK]

Common Mistakes:

• Using GET instead of POST to register
• Confusing schema formats (JSON vs XML/YAML/CSV)
• Using wrong API endpoints

3. Given this code snippet using Kafka Schema Registry client, what will be the output?

schema = '{"type":"record","name":"User","fields":[{"name":"name","type":"string"}]}'
subject = "User-value"
schema_id = client.register_schema(subject, schema)
print(schema_id)

medium

A. An integer ID representing the registered schema

B. The original schema string printed

C. A JSON object with schema details

D. A runtime error due to missing client method

Solution

1. Step 1: Understand the register_schema method

   This method registers the schema and returns an integer ID assigned by Schema Registry.

2. Step 2: Analyze the print statement

   Printing schema_id outputs the integer ID, not the schema string or JSON object.

3. Final Answer:

   An integer ID representing the registered schema -> Option A

4. Quick Check:

   Register returns schema ID integer [OK]

Hint: Register returns schema ID number, not schema text [OK]

Common Mistakes:

• Expecting schema string instead of ID
• Confusing output with JSON object
• Assuming method does not exist

4. Identify the error in this Kafka Schema Registry usage snippet:

schema = '{"type":"record","name":"User","fields":[{"name":"age","type":"int"}]}'
subject = "User-value"
schema_id = client.register_schema(subject, schema)
client.register_schema(subject, schema)

medium

A. Registering the same schema twice causes an error

B. No error; registering same schema returns same ID

C. Schema string is not valid JSON

D. Missing schema version in registration call

Solution

1. Step 1: Check schema validity

   The schema string is valid JSON describing a record with an int field.

2. Step 2: Understand Schema Registry behavior on duplicate schemas

   Registering the same schema again returns the same schema ID without error.

3. Final Answer:

   No error; registering same schema returns same ID -> Option B

4. Quick Check:

   Duplicate schema registration returns same ID [OK]

Hint: Duplicate schema registration is safe and returns same ID [OK]

Common Mistakes:

• Assuming duplicate registration causes error
• Thinking schema string is invalid JSON
• Expecting version parameter in register call

5. You want to evolve a Kafka message schema by adding a new optional field without breaking existing consumers. Which Schema Registry feature helps achieve this safely?

hard

A. Schema compatibility settings (e.g., BACKWARD compatibility)

B. Schema encryption for secure transmission

C. Schema version rollback to previous schema

D. Schema validation disabling for faster registration

Solution

1. Step 1: Understand schema evolution needs

   Adding a new optional field requires compatibility so old consumers still work.

2. Step 2: Identify Schema Registry feature for compatibility

   Schema compatibility settings like BACKWARD ensure new schemas don't break old consumers.

3. Step 3: Eliminate other options

   Encryption, rollback, or disabling validation do not help safe schema evolution.

4. Final Answer:

   Schema compatibility settings (e.g., BACKWARD compatibility) -> Option A

5. Quick Check:

   Compatibility settings enable safe schema evolution [OK]

Hint: Use compatibility settings to evolve schemas safely [OK]

Common Mistakes:

• Confusing encryption with compatibility
• Thinking rollback helps forward compatibility
• Disabling validation breaks schema safety