Connector configuration in Kafka - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Kafka connector configuration validation?

Given the following Kafka connector configuration JSON, what will be the result of the connector validation?

Kafka

{
  "name": "my-source-connector",
  "config": {
    "connector.class": "FileStreamSource",
    "tasks.max": "2",
    "file": "/tmp/test.txt",
    "topic": "test-topic"
  }
}

A Validation failed: 'file' property path does not exist.

B Validation failed: 'tasks.max' must be an integer, not a string.

C Validation failed: Missing required property 'topic'.

D Validation successful: Connector configuration is valid.

💡 Hint

Check the data types of the configuration properties.

The 'tasks.max' property can be provided as a string and will be parsed as an integer by Kafka Connect. All required properties are present and valid.

❓ Predict Output

intermediate

What error does this Kafka sink connector configuration produce?

Analyze the following sink connector configuration and determine the error it will cause when deployed.

Kafka

{
  "name": "my-sink-connector",
  "config": {
    "connector.class": "FileStreamSink",
    "tasks.max": 1,
    "file": 12345,
    "topics": "my-topic"
  }
}

A Validation failed: 'topics' property must be 'topic' (singular).

B Validation failed: 'file' property must be a string path, not a number.

C No error, configuration is valid.

D Validation failed: 'tasks.max' must be a string, not an integer.

💡 Hint

Check the data types of each property carefully.

The 'file' property must be a string representing the file path. Providing a number causes a validation error.

🔧 Debug

advanced

Why does this Kafka connector fail to start?

This connector configuration fails to start. Identify the cause.

Kafka

{
  "name": "json-source-connector",
  "config": {
    "connector.class": "JsonFileSource",
    "tasks.max": 3,
    "file": "/data/input.json",
    "topic": "json-topic",
    "value.converter": "org.apache.kafka.connect.json.JsonConverter",
    "value.converter.schemas.enable": "false"
  }
}

A The 'tasks.max' value cannot be greater than 2 for this connector.

B The 'value.converter.schemas.enable' must be a boolean true/false, not a string.

C The connector class 'JsonFileSource' does not exist in Kafka Connect.

D The 'topic' property is misspelled; it should be 'topics'.

💡 Hint

Check if the connector class is a valid built-in or custom connector.

Kafka Connect does not have a built-in connector named 'JsonFileSource'. This causes the connector to fail at startup.

📝 Syntax

advanced

Which configuration JSON is syntactically correct for a Kafka connector?

Select the syntactically correct Kafka connector configuration JSON.

A

{
  "name": "my-connector",
  "config": {
    "connector.class": "FileStreamSource",
    "tasks.max": 1,
    "file": "/tmp/input.txt",
    "topic": "my-topic"
  }
}

B

{
  "name": "my-connector",
  "config": {
    "connector.class": "FileStreamSource",
    "tasks.max": "one",
    "file": "/tmp/input.txt",
    "topic": "my-topic"
  }
}

C

}
}  
"cipot-ym" :"cipot"    
,"txt.tupni/pmt/" :"elif"    
,1 :"xam.sksat"    
,"ecruoSmaertSeliF" :"ssalc.rotcennoc"    
{ :"gifnoc"  
,"rotcennoc-ym" :"eman"  
{

D

{
  "name": "my-connector",
  "config": {
    "connector.class": "FileStreamSource",
    "tasks.max": 1
    "file": "/tmp/input.txt",
    "topic": "my-topic"
  }
}

💡 Hint

Look for missing commas, trailing commas, and correct data types.

Option A is valid JSON with correct commas and data types. Option A misses a comma after 'tasks.max'. Option A has a trailing comma after the last property. Option A uses a string 'one' instead of an integer for 'tasks.max'.

🚀 Application

expert

How many tasks will this Kafka connector run?

Given this connector configuration, how many tasks will Kafka Connect run?

Kafka

{
  "name": "multi-task-connector",
  "config": {
    "connector.class": "FileStreamSource",
    "tasks.max": 4,
    "file": "/var/log/app.log",
    "topic": "app-logs"
  }
}

A Only 1 task will run because the connector supports a single task.

B The number of tasks is decided by the number of partitions in the topic, ignoring 'tasks.max'.

C Kafka Connect will run 0 tasks because 'file' path is invalid.

D Exactly 4 tasks will run as specified by 'tasks.max'.

💡 Hint

Consider the connector type and its support for parallel tasks.

The FileStreamSource connector supports only a single task regardless of 'tasks.max' value. So only 1 task will run.

Practice

1. What is the main purpose of a connector configuration in Kafka Connect?

easy

A. To specify how data moves between Kafka and external systems

B. To define the Kafka broker's internal settings

C. To configure Kafka topic partitions

D. To set the Kafka consumer group ID

Solution

1. Step 1: Understand Kafka Connect role

   Kafka Connect uses connectors to move data between Kafka and other systems.

2. Step 2: Identify configuration purpose

   The connector configuration tells Kafka Connect how to perform this data movement.

3. Final Answer:

   To specify how data moves between Kafka and external systems -> Option A

4. Quick Check:

   Connector configuration = data movement setup [OK]

Hint: Connector config defines data flow direction [OK]

Common Mistakes:

• Confusing connector config with Kafka broker settings
• Thinking it configures Kafka topic partitions
• Mixing up consumer group ID with connector config

2. Which of the following is the correct way to specify the connector class in a Kafka Connect configuration JSON?

easy

A. "connectorClassName": "org.apache.kafka.connect.file.FileStreamSourceConnector"

B. "connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector"

C. "connectorClass": "org.apache.kafka.connect.file.FileStreamSourceConnector"

D. "connector-class": "org.apache.kafka.connect.file.FileStreamSourceConnector"

Solution

1. Step 1: Recall Kafka Connect JSON key format

   Kafka Connect uses keys with dot notation, e.g., "connector.class".

2. Step 2: Match correct key name

   Only "connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector" uses the correct key "connector.class" with quotes and colon.

3. Final Answer:

   "connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector" -> Option B

4. Quick Check:

   Correct key is "connector.class" [OK]

Hint: Use dot notation keys like "connector.class" [OK]

Common Mistakes:

• Using hyphens instead of dots in keys
• Using camelCase keys instead of dot notation
• Omitting quotes around keys or values

3. Given this connector configuration snippet:

{
  "name": "my-source-connector",
  "connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector",
  "tasks.max": "2",
  "file": "/tmp/input.txt",
  "topic": "my-topic"
}

What does the tasks.max setting control?

medium

A. Maximum number of Kafka topics to consume

B. Maximum number of retries for failed tasks

C. Maximum size of each task's data batch

D. Maximum number of tasks to run in parallel for this connector

Solution

1. Step 1: Understand tasks.max meaning

   In Kafka Connect, tasks.max sets how many parallel tasks the connector can run.

2. Step 2: Eliminate other options

   It does not control topic count, batch size, or retry count.

3. Final Answer:

   Maximum number of tasks to run in parallel for this connector -> Option D

4. Quick Check:

   tasks.max = parallel task limit [OK]

Hint: tasks.max limits parallel connector tasks [OK]

Common Mistakes:

• Confusing tasks.max with topic count
• Thinking it controls batch size
• Assuming it sets retry attempts

4. You have this connector config JSON:

{
  "name": "my-sink-connector",
  "connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
  "tasks.max": 1,
  "topics": "my-topic"
}

What is wrong with this configuration?

medium

A. "connector.class" value is incorrect for a sink connector

B. "topics" key should be "topic" (singular)

C. Missing "file" property to specify output file

D. "tasks.max" should be a string, not a number

Solution

1. Step 1: Check required properties for FileStreamSinkConnector

   This connector needs a "file" property to specify the output file path.

2. Step 2: Validate other properties

   "tasks.max" can be a number or string, "topics" is valid plural, and "connector.class" is correct.

3. Final Answer:

   Missing "file" property to specify output file -> Option C

4. Quick Check:

   FileStreamSinkConnector requires "file" property [OK]

Hint: FileStreamSinkConnector needs "file" path set [OK]

Common Mistakes:

• Assuming "tasks.max" must be string
• Confusing "topic" and "topics" keys
• Changing connector.class unnecessarily

5. You want to configure a Kafka Connect source connector to read from a database table and write to a Kafka topic. Which configuration property is essential to specify the Kafka topic where data will be published?

hard

A. "topic"

B. "topics"

C. "topics.regex"

D. "kafka.topic"

Solution

1. Step 1: Understand topic properties in connectors

   Source connectors typically use "topic" (singular) to specify the Kafka topic to publish data to.

2. Step 2: Differentiate from other keys

   "topics" is common for sink connectors; "topics.regex" is for pattern matching; "kafka.topic" is not standard.

3. Final Answer:

   "topic" -> Option A

4. Quick Check:

   Use "topic" to specify Kafka topic for sources [OK]

Hint: Use "topic" for source connector Kafka topic [OK]

Common Mistakes:

• Using plural "topics" instead of "topic"
• Confusing "topics.regex" with direct topic specification
• Inventing non-standard keys like "kafka.topic"