Avro schema definition in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Avro schema validation?

Given this Avro schema and a JSON record, what will be the result of validating the record against the schema?

Kafka

{
  "type": "record",
  "name": "User",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "age", "type": "int"}
  ]
}

Record:
{"name": "Alice", "age": "25"}

A Validation fails due to age being a string instead of int

B Validation passes successfully

C Validation fails due to missing required field

D Validation passes but age is converted to int automatically

💡 Hint

Check the data types of each field in the record compared to the schema.

The schema expects 'age' to be an integer, but the record provides it as a string. Avro does not automatically convert types, so validation fails.

🧠 Conceptual

intermediate

Which Avro schema type is best for optional fields?

You want to define a field in an Avro schema that may or may not be present in the data. Which schema type should you use?

A "null"

B ["null", "type"] (a union with null first)

C "nullable"

D "optional"

💡 Hint

Avro uses unions to represent optional fields.

In Avro, optional fields are represented by a union of null and the actual type, with null usually first to indicate the field can be missing.

🔧 Debug

advanced

Why does this Avro schema cause a parsing error?

Identify the error in this Avro schema snippet: { "type": "record", "name": "Product", "fields": [ {"name": "id", "type": "int"}, {"name": "price", "type": ["float", "null"]}, {"name": "tags", "type": "array", "items": "string"} ] }

A The 'items' property should be inside the 'type' array for 'tags'

B The 'type' for 'tags' should be "array" with 'items' as a sibling key, but here 'items' is misplaced

C The 'type' for 'tags' should be an object with 'type' and 'items' keys, not a string

D The 'items' key is correctly placed; the error is in the union type for 'price'

💡 Hint

Check the structure of the 'array' type in Avro schemas.

In Avro, when defining an array type, the 'type' must be an object with 'type': 'array' and an 'items' key. Here, 'type' is a string and 'items' is a sibling key, which is invalid.

📝 Syntax

advanced

Which Avro schema snippet is syntactically correct for a nested record?

Choose the correct Avro schema snippet defining a record with a nested record field.

A

{
  "type": "record",
  "name": "Employee",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "address", "type": "record",
      "name": "Address",
      "fields": [
        {"name": "street", "type": "string"},
        {"name": "city", "type": "string"}
      ]
    }
  ]
}

B

{
  "type": "record",
  "name": "Employee",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "address", "type": "record",
      "fields": [
        {"name": "street", "type": "string"},
        {"name": "city", "type": "string"}
      ]
    }
  ]
}

C

{
  "type": "record",
  "name": "Employee",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "address", "type": ["null", {
      "type": "record",
      "name": "Address",
      "fields": [
        {"name": "street", "type": "string"},
        {"name": "city", "type": "string"}
      ]
    }]}
  ]
}

D

{
  "type": "record",
  "name": "Employee",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "address", "type": {
      "type": "record",
      "name": "Address",
      "fields": [
        {"name": "street", "type": "string"},
        {"name": "city", "type": "string"}
      ]
    }}
  ]
}

💡 Hint

Nested records must be defined as an object with type 'record' and a name.

Option D correctly defines the nested record as an object with 'type': 'record', a 'name', and 'fields'. Option D and D miss the nested record object structure. Option D is valid but includes a union with null, which is optional but not required here.

🚀 Application

expert

How many fields will the following Avro schema record have after default values are applied?

Consider this Avro schema: { "type": "record", "name": "Event", "fields": [ {"name": "id", "type": "string"}, {"name": "timestamp", "type": "long"}, {"name": "metadata", "type": ["null", "string"], "default": null}, {"name": "priority", "type": "int", "default": 5} ] } If a record is created with only 'id' and 'timestamp' provided, how many fields will the record have after applying defaults?

A 4 fields

B 3 fields

C 5 fields

D 2 fields

💡 Hint

Fields with defaults are added automatically if missing.

The record has 4 fields total. If only 'id' and 'timestamp' are provided, the fields 'metadata' and 'priority' will be added with their default values, making 4 fields in total.

Practice

1. What is the main purpose of an Avro schema in Kafka?

easy

A. To route messages to different topics

B. To define the structure and data types of messages

C. To compress messages for faster transmission

D. To encrypt messages for security

Solution

1. Step 1: Understand Avro schema role

   An Avro schema describes how data is organized and typed in messages.

2. Step 2: Compare options with schema purpose

   Only defining structure and data types matches the schema's role.

3. Final Answer:

   To define the structure and data types of messages -> Option B

4. Quick Check:

   Avro schema = data structure definition [OK]

Hint: Avro schema describes message format, not security or routing [OK]

Common Mistakes:

• Confusing schema with encryption or compression
• Thinking schema controls message routing
• Assuming schema is for message storage

2. Which of the following is the correct way to start an Avro record schema definition?

easy

A. {"type": "enum", "symbols": ["A", "B"]}

B. {"type": "array", "items": "string"}

C. {"type": "map", "values": "int"}

D. {"type": "record", "name": "User", "fields": []}

Solution

1. Step 1: Identify record schema syntax

   An Avro record schema starts with type "record", a name, and a fields array.

2. Step 2: Match options to record schema

   Only {"type": "record", "name": "User", "fields": []} matches the record schema format correctly.

3. Final Answer:

   {"type": "record", "name": "User", "fields": []} -> Option D

4. Quick Check:

   Record schema = type "record" with name and fields [OK]

Hint: Record schemas always start with type "record" and have fields [OK]

Common Mistakes:

• Using array or map types instead of record
• Missing the name or fields keys
• Confusing enum with record

3. Given this Avro schema snippet:

{"type": "record", "name": "Person", "fields": [{"name": "age", "type": "int"}]}

What is the expected type of the age field in messages?

medium

A. int

B. boolean

C. string

D. float

Solution

1. Step 1: Read the field definition

   The field "age" has type "int" in the schema.

2. Step 2: Determine expected data type

   Messages must have an integer value for "age" as per schema.

3. Final Answer:

   int -> Option A

4. Quick Check:

   Field type "int" means integer value expected [OK]

Hint: Field type in schema directly shows expected data type [OK]

Common Mistakes:

• Assuming string type by default
• Confusing int with float or boolean
• Ignoring the schema field type

4. Identify the error in this Avro schema snippet:

{"type": "record", "name": "Product", "fields": [{"name": "price", "type": "double"}, {"name": "available"}]}

medium

A. Missing type for field "available"

B. Invalid record name

C. Fields array should be empty

D. Type "double" is not allowed

Solution

1. Step 1: Check each field for required keys

   Each field must have a "name" and a "type" key.

2. Step 2: Find missing keys

   The field "available" lacks a "type" key, causing an error.

3. Final Answer:

   Missing type for field "available" -> Option A

4. Quick Check:

   Every field needs a type [OK]

Hint: Every field must have a type key defined [OK]

Common Mistakes:

• Forgetting to add type for all fields
• Thinking record name can be empty
• Assuming double type is invalid

5. You want to add an optional field email of type string with a default empty value to an Avro record schema. Which field definition is correct?

hard

A. {"name": "email", "type": "string"}

B. {"name": "email", "type": ["null", "string"], "default": ""}

C. {"name": "email", "type": ["string", "null"], "default": ""}

D. {"name": "email", "type": ["null", "string"], "default": null}

Solution

1. Step 1: Understand optional field in Avro

   Optional fields use a union type with "null" and the actual type, with a default value matching the first type in the union.

2. Step 2: Check default value and type order

   For default empty string "", the first type must be "string" followed by "null"; {"name": "email", "type": ["string", "null"], "default": ""} matches.

3. Final Answer:

   {"name": "email", "type": ["string", "null"], "default": ""} -> Option C

4. Quick Check:

   Optional with default "" = ["string", "null"] union [OK]

Hint: Union first type must match default; include null for optional [OK]

Common Mistakes:

• Not using union with null for optional fields
• Using null first with string default (invalid)
• Setting default to null when empty string needed