Avro schema definition in Kafka - Step-by-Step Execution

Process Flow - Avro schema definition

Start: Define schema fields

↓

Specify field names and types

↓

Set record name and namespace

↓

Add optional default values

↓

Validate schema format

↓

Use schema for serialization/deserialization

↓

End

This flow shows how to create an Avro schema by defining fields, setting metadata, validating, and then using it for data encoding.

Execution Sample

Kafka

 {
  "type": "record",
  "name": "User",
  "fields": [
    {"name": "id", "type": "int"},
    {"name": "name", "type": "string"}
  ]
}

Defines a simple Avro schema for a User record with id and name fields.

Process Table

Step	Action	Field/Property	Value	Result
1	Define schema type	"type"	"record"	Schema type set to record
2	Set record name	"name"	"User"	Record named User
3	Add field	"fields[0]"	{"name": "id", "type": "int"}	Field id of type int added
4	Add field	"fields[1]"	{"name": "name", "type": "string"}	Field name of type string added
5	Validate schema	Full schema	Valid JSON with required keys	Schema is valid
6	Use schema	Serialization	Encode User data	Data serialized using schema
7	Use schema	Deserialization	Decode User data	Data deserialized back to User object

💡 All fields defined and schema validated; ready for use in Kafka serialization.

Status Tracker

Variable	Start	After Step 3	After Step 4	Final
schema	{}	{"type": "record", "name": "User", "fields": [{"name": "id", "type": "int"}]}	{"type": "record", "name": "User", "fields": [{"name": "id", "type": "int"}, {"name": "name", "type": "string"}]}	Complete schema object

Key Moments - 3 Insights

Why do we need to specify the "type" as "record" at the start?

The "type": "record" tells Avro this schema defines a structured object with named fields, not just a simple type. See execution_table step 1.

Can fields be added without a name or type?

No, each field must have a "name" and a "type" to be valid. Missing these causes schema validation to fail. See execution_table steps 3 and 4.

What happens if the schema is not valid JSON?

The schema cannot be used for serialization or deserialization until fixed. Validation step (5) ensures the schema is correct.

Visual Quiz - 3 Questions

Look at the execution_table, what is the value of the "name" property after step 2?

A "record"

B "User"

C "id"

D "fields"

💡 Hint

Check the 'Field/Property' and 'Value' columns at step 2 in execution_table.

At which step is the second field "name" added to the schema?

A Step 1

B Step 3

C Step 4

D Step 5

💡 Hint

Look for the action 'Add field' and field details in execution_table rows.

If the "type" property was missing from a field, what would happen during validation?

A Schema validation would fail

B Field would default to string type

C Schema would still be valid

D Kafka would ignore the field

💡 Hint

Refer to key_moments about required field properties and validation step.

Concept Snapshot

Avro schema defines data structure for Kafka.
Use "type": "record" for objects.
List fields with "name" and "type".
Validate schema before use.
Schema enables data serialization/deserialization.

Full Transcript

This visual execution shows how to define an Avro schema for Kafka. First, set the schema type to 'record' to indicate a structured object. Then, assign a name to the record, like 'User'. Next, add fields one by one, each with a name and a type, such as 'id' as int and 'name' as string. After defining all fields, validate the schema to ensure it is correct JSON and has all required parts. Once validated, the schema can be used to serialize data into bytes for Kafka and deserialize bytes back into objects. Key points include always specifying 'type' and 'name' for fields and validating the schema before use. The execution table tracks each step, showing how the schema object grows and is checked. The variable tracker shows the schema's state after each field addition. This step-by-step helps beginners see how Avro schemas are built and used in Kafka.