JSON Schema and Protobuf support in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is JSON Schema in Kafka?

JSON Schema is a way to define the structure, types, and validation rules for JSON data used in Kafka messages. It helps ensure data consistency and compatibility between producers and consumers.

beginner

What is Protobuf and why is it used with Kafka?

Protobuf (Protocol Buffers) is a compact, efficient binary serialization format developed by Google. It is used with Kafka to serialize structured data for fast transmission and smaller message sizes compared to JSON.

intermediate

How does Kafka Schema Registry support JSON Schema and Protobuf?

Kafka Schema Registry stores and manages schemas for JSON Schema and Protobuf. It validates messages against schemas, supports schema evolution, and enables compatibility checks to avoid breaking changes.

intermediate

What is schema evolution and how does it relate to JSON Schema and Protobuf in Kafka?

Schema evolution means updating schemas over time without breaking existing data consumers. JSON Schema and Protobuf support schema evolution by allowing backward and forward compatible changes, managed by Kafka Schema Registry.

advanced

What are the advantages of using Protobuf over JSON Schema in Kafka?

Protobuf messages are smaller and faster to serialize/deserialize than JSON. Protobuf enforces strict typing and supports efficient binary encoding, which reduces network bandwidth and improves performance in Kafka pipelines.

What does Kafka Schema Registry do with JSON Schema and Protobuf?

A Only stores JSON data

B Converts JSON to XML

C Stores, validates, and manages schemas

D Encrypts Kafka messages

Kafka Schema Registry stores and manages schemas for JSON Schema and Protobuf, validating messages and supporting schema evolution.

Which serialization format is more compact and faster in Kafka?

A JSON Schema

B Protobuf

C XML

D CSV

Protobuf uses binary encoding which is more compact and faster than JSON Schema's text-based format.

What is schema evolution in Kafka?

A Changing schemas without breaking compatibility

B Deleting old schemas

C Encrypting schemas

D Converting schemas to JSON

Schema evolution allows updating schemas while maintaining compatibility with existing data consumers.

Which of these is NOT a benefit of using JSON Schema with Kafka?

A Compact binary encoding

B Human-readable format

C Schema compatibility checks

D Data validation

JSON Schema uses text-based JSON format, not compact binary encoding like Protobuf.

Why use Kafka Schema Registry with Protobuf?

A To encrypt Protobuf messages

B To convert Protobuf to JSON automatically

C To store messages permanently

D To manage and validate Protobuf schemas centrally

Kafka Schema Registry centrally manages and validates Protobuf schemas to ensure compatibility and correctness.

Explain how Kafka Schema Registry supports both JSON Schema and Protobuf and why this is important.

Think about how managing schemas helps avoid errors and data mismatches.

Describe the differences between JSON Schema and Protobuf when used with Kafka.

Consider format, performance, and readability.

Practice

1. What is the main purpose of using JSON Schema or Protobuf with Kafka messages?

easy

A. To define the structure and format of messages so producers and consumers agree

B. To encrypt Kafka messages for security

C. To speed up Kafka message delivery by skipping validation

D. To store Kafka messages permanently in a database

Solution

1. Step 1: Understand message format agreement

   JSON Schema and Protobuf define how messages look so both sender and receiver understand the data.

2. Step 2: Identify the main purpose

   This agreement prevents errors and confusion when reading messages.

3. Final Answer:

   To define the structure and format of messages so producers and consumers agree -> Option A

4. Quick Check:

   Message format agreement = C [OK]

Hint: They define message format so systems agree on data [OK]

Common Mistakes:

• Thinking JSON Schema encrypts messages
• Believing Protobuf speeds delivery by skipping validation
• Confusing schema with storage

2. Which of the following is the correct way to specify a Protobuf schema for a Kafka message?

easy

A. { 'name': 'string', 'id': 'int' }

B. schema { field name: string; field id: int; }

C. stringint

D. message Person { string name = 1; int32 id = 2; }

Solution

1. Step 1: Recognize Protobuf syntax

   Protobuf uses the 'message' keyword with fields and their types and numbers.

2. Step 2: Match the correct syntax

   message Person { string name = 1; int32 id = 2; } shows correct Protobuf message definition syntax.

3. Final Answer:

   message Person { string name = 1; int32 id = 2; } -> Option D

4. Quick Check:

   Protobuf syntax = B [OK]

Hint: Protobuf uses 'message' keyword and numbered fields [OK]

Common Mistakes:

• Confusing JSON or XML syntax with Protobuf
• Missing field numbers in Protobuf
• Using invalid keywords like 'schema'

3. Given this JSON Schema snippet for a Kafka message:

{
  "type": "object",
  "properties": {
    "userId": {"type": "string"},
    "age": {"type": "integer"}
  },
  "required": ["userId"]
}

Which message is valid according to this schema?

medium

A. {"userId": "abc123", "age": 30}

B. {"age": 30}

C. {"userId": 123, "age": 30}

D. {"userId": "abc123", "age": "30"}

Solution

1. Step 1: Check required fields and types

   The schema requires 'userId' as a string and 'age' as an integer (optional).

2. Step 2: Validate each option

   {"userId": "abc123", "age": 30} has 'userId' as string and 'age' as integer, so it is valid.

3. Final Answer:

   {"userId": "abc123", "age": 30} -> Option A

4. Quick Check:

   Required string userId and integer age = A [OK]

Hint: Required fields must be present with correct types [OK]

Common Mistakes:

• Missing required 'userId' field
• Using wrong data types for fields
• Confusing string and integer types

4. You have this Protobuf schema:

message Event {
  string id = 1;
  int32 count = 2;
}

Which of the following is CORRECT about using it with Kafka?

medium

A. Using 'int32' instead of 'integer' causes error

B. Field numbers must be unique and start from 1

C. Missing a root 'schema' keyword causes failure

D. Field names cannot be strings

Solution

1. Step 1: Check Protobuf field numbering rules

   Field numbers must be unique and start from 1, which is true here.

2. Step 2: Identify the correct statement

   Field numbers must be unique and start from 1 correctly describes the field numbering rule. Options B, C, D are incorrect statements.

3. Final Answer:

   Field numbers must be unique and start from 1 -> Option B

4. Quick Check:

   Protobuf field numbering rule = A [OK]

Hint: Field numbers must be unique and start at 1 [OK]

Common Mistakes:

• Thinking 'int32' is invalid
• Expecting a 'schema' keyword in Protobuf
• Confusing field names with types

5. You want to optimize Kafka message size and speed. Which schema choice is best and why?

hard

A. Use JSON Schema because it is human-readable and easy to debug

B. Use XML schema because it supports complex data types

C. Use Protobuf because it is compact and faster to serialize/deserialize

D. Use plain text messages without schema for maximum speed

Solution

1. Step 1: Compare JSON Schema and Protobuf

   JSON Schema is readable but larger; Protobuf is compact and faster.

2. Step 2: Choose based on optimization goal

   For smaller size and speed, Protobuf is better.

3. Final Answer:

   Use Protobuf because it is compact and faster to serialize/deserialize -> Option C

4. Quick Check:

   Compact and fast = D [OK]

Hint: Protobuf is compact and faster than JSON Schema [OK]

Common Mistakes:

• Choosing JSON Schema for speed optimization
• Ignoring schema benefits and using plain text
• Confusing XML with Protobuf advantages