Transform and converter chains in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a Kafka Connect transform chain

Consider a Kafka Connect source connector with the following transform chain applied to each record:

{"transforms": "dropField,replaceValue", "transforms.dropField.type": "org.apache.kafka.connect.transforms.ReplaceField$Value", "transforms.dropField.blacklist": "password", "transforms.replaceValue.type": "org.apache.kafka.connect.transforms.ReplaceField$Value", "transforms.replaceValue.whitelist": "username"}

Given a record with value:

{"username": "alice", "password": "secret", "email": "[e-mail]"}

What will be the value of the record after the transform chain?

A {"password": "secret"}

B {"username": "alice", "email": "[e-mail]"}

C {"email": "[e-mail]"}

D {"username": "alice"}

💡 Hint

Think about the order of transforms and what each one keeps or removes.

The first transform drops the 'password' field, so the record becomes {"username": "alice", "email": "[e-mail]"}. The second transform keeps only the 'username' field, removing 'email'. So the final value is {"username": "alice"}.

❓ Predict Output

intermediate

Result of converter chain with JSON and Avro

A Kafka Connect sink connector uses a converter chain where the key converter is JSON and the value converter is Avro. The source sends a record with key:

{"id": 123}

and value:

{"name": "Bob", "age": 30}

What is the expected format of the key and value when the sink receives the record?

A Key as Avro bytes, value as JSON bytes

B Both key and value as Avro bytes

C Key as JSON bytes, value as Avro bytes

D Both key and value as JSON bytes

💡 Hint

Remember the converter chain applies separately to key and value.

The key converter is JSON, so the key is serialized as JSON bytes. The value converter is Avro, so the value is serialized as Avro bytes.

🔧 Debug

advanced

Identify the error in transform chain configuration

A user configures a Kafka Connect transform chain as follows:

{"transforms": "maskField", "transforms.maskField.type": "org.apache.kafka.connect.transforms.MaskField$Value", "transforms.maskField.fields": "ssn"}

But the connector fails to start with an error about transform class not found. What is the most likely cause?

A The transform chain name 'maskField' is reserved and cannot be used

B The transform class name is incorrect; it should be MaskField$Key or MaskField$Value with correct casing

C The 'fields' property is missing a comma-separated list of fields

D The connector requires a converter to be set before transforms

💡 Hint

Check the exact class name spelling and casing for the transform.

The class name is case-sensitive and must match exactly. 'MaskField$Value' is correct, but if the casing or spelling is wrong, the connector cannot find the class.

🧠 Conceptual

advanced

Understanding transform chain order impact

In Kafka Connect, you have two transforms in a chain:
1. ReplaceField to whitelist only 'user' and 'email'
2. MaskField to mask the 'email' field

If you reverse the order of these transforms, what is the impact on the final record?

A Reversing the order causes the MaskField to fail because 'email' is removed before masking

B Reversing the order has no impact; the final record is the same

C Reversing the order causes the 'user' field to be masked instead of 'email'

D Reversing the order causes the connector to crash due to invalid transform chain

💡 Hint

Think about which fields exist when each transform runs.

If ReplaceField runs first and removes all fields except 'user' and 'email', then MaskField can mask 'email'. If MaskField runs first, it tries to mask 'email' before ReplaceField removes other fields. But if ReplaceField runs second and removes 'email', the mask has no effect. Reversing the order can cause MaskField to fail if the field is missing.

❓ Predict Output

expert

Output of complex converter and transform chain

A Kafka Connect source connector uses the following configuration:

{
  "key.converter": "org.apache.kafka.connect.json.JsonConverter",
  "key.converter.schemas.enable": false,
  "value.converter": "io.confluent.connect.avro.AvroConverter",
  "value.converter.schema.registry.url": "http://localhost:8081",
  "transforms": "unwrap,dropFields",
  "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
  "transforms.dropFields.type": "org.apache.kafka.connect.transforms.ReplaceField$Value",
  "transforms.dropFields.blacklist": "metadata,internal"
}

Given a source record value:

{"before": null, "after": {"id": 1, "name": "John", "metadata": "info", "internal": "secret"}}

What will be the value passed to the sink after the transform chain?

A {"id": 1, "name": "John"}

B {"before": null, "after": {"id": 1, "name": "John"}}

C {"id": 1, "name": "John", "metadata": "info", "internal": "secret"}

D {"metadata": "info", "internal": "secret"}

💡 Hint

The unwrap transform extracts the 'after' field, then dropFields removes blacklisted fields.

The unwrap transform extracts the 'after' object, so the record becomes {"id": 1, "name": "John", "metadata": "info", "internal": "secret"}. Then the dropFields transform removes 'metadata' and 'internal', leaving {"id": 1, "name": "John"}.