Avro schema definition in Kafka - Mini Project: Build & Apply

Avro Schema Definition for Kafka

📖 Scenario: You are working on a Kafka project where you need to define a data schema using Avro. This schema will describe the structure of messages sent to a Kafka topic about user information.

🎯 Goal: Create an Avro schema that defines a record for user data with specific fields and types.

📋 What You'll Learn

Define an Avro schema as a JSON string

Include a record named User

Add fields name (string), age (int), and email (string)

Set the namespace to com.example.kafka

💡 Why This Matters

🌍 Real World

Avro schemas are used to define the structure of data sent through Kafka topics, ensuring producers and consumers agree on the data format.

💼 Career

Understanding Avro schema definition is important for roles involving data engineering, streaming data pipelines, and Kafka-based system development.

1

Create the basic Avro schema structure

Create a variable called user_schema and assign it a JSON string that starts with { and includes "type": "record" and "name": "User".

Kafka

# Your code here

Need a hint?

Start by creating a string that looks like a JSON object with type and name keys.

2

Add the namespace to the schema

Update the user_schema string to include the "namespace": "com.example.kafka" key-value pair inside the JSON object.

Kafka

user_schema = ''
# Your code here

Need a hint?

Add the namespace key with the exact value inside the JSON string.

3

Add fields to the Avro schema

Modify the user_schema string to include a "fields" array with three fields: {"name": "name", "type": "string"}, {"name": "age", "type": "int"}, and {"name": "email", "type": "string"}.

Kafka

user_schema = ''
# Your code here

Need a hint?

Remember to format the fields as a JSON array inside the string.

4

Print the complete Avro schema

Write a print statement to display the user_schema variable.

Kafka

print()
# Your code here

Need a hint?

Use print(user_schema) to show the schema string.