Source connectors in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to specify the connector class for a source connector.

Kafka

{
  "name": "my-source-connector",
  "connector.class": "[1]",
  "tasks.max": "1"
}

Drag options to blanks, or click blank then click option'

AJdbcSinkConnector

BFileStreamSinkConnector

CFileStreamSourceConnector

DConsoleSinkConnector

Attempts:

3 left

💡 Hint

Common Mistakes

Using a sink connector class instead of a source connector class.

Confusing source and sink connectors.

The connector.class property defines the connector type. For a source connector reading from a file, FileStreamSourceConnector is correct.

2fill in blank

medium

Complete the code to set the Kafka topic where the source connector will send data.

Kafka

{
  "name": "my-source-connector",
  "connector.class": "FileStreamSourceConnector",
  "tasks.max": "1",
  "topic": "[1]"
}

Drag options to blanks, or click blank then click option'

Ainput-topic

Boutput-topic

Csource-topic

Dsink-topic

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'sink-topic' which is usually for sink connectors.

Confusing source and sink topic names.

The topic property defines the Kafka topic where the source connector sends data. 'input-topic' is a common name for such a topic.

3fill in blank

hard

Fix the error in the connector configuration key for the file path.

Kafka

{
  "name": "my-source-connector",
  "connector.class": "FileStreamSourceConnector",
  "tasks.max": "1",
  "topic": "input-topic",
  "file[1]": "/tmp/test.txt"
}

Drag options to blanks, or click blank then click option'

A_filepath

B_path

C.filepath

D.path

Attempts:

3 left

💡 Hint

Common Mistakes

Using dot notation instead of underscore in property keys.

Guessing property names without checking documentation.

The correct property key for the file path in FileStreamSourceConnector is file_path using an underscore.

4fill in blank

hard

Fill both blanks to configure a JDBC source connector with the correct class and mode.

Kafka

{
  "name": "jdbc-source-connector",
  "connector.class": "[1]",
  "mode": "[2]",
  "tasks.max": "1"
}

Drag options to blanks, or click blank then click option'

Aio.confluent.connect.jdbc.JdbcSourceConnector

Bio.confluent.connect.jdbc.JdbcSinkConnector

Cincrementing

Dbulk

Attempts:

3 left

💡 Hint

Common Mistakes

Using the sink connector class instead of the source connector class.

Confusing mode values like 'bulk' and 'incrementing'.

The JDBC source connector class is io.confluent.connect.jdbc.JdbcSourceConnector. The mode can be set to incrementing to fetch new rows based on an incrementing column.

5fill in blank

hard

Fill all three blanks to create a source connector configuration that reads from a file and sends to a topic.

Kafka

{
  "name": "file-source-connector",
  "connector.class": "[1]",
  "file[2]": "/var/log/app.log",
  "topic": "[3]",
  "tasks.max": "1"
}

Drag options to blanks, or click blank then click option'

AFileStreamSourceConnector

B_path

Capp-logs

DFileStreamSinkConnector

Attempts:

3 left

💡 Hint

Common Mistakes

Using sink connector class instead of source.

Using dot instead of underscore in property keys.

Choosing unrelated topic names.

The source connector class is FileStreamSourceConnector. The file path property uses underscore as file_path. The topic name app-logs is a suitable Kafka topic for logs.

Practice

1. What is the main purpose of a Kafka source connector?

easy

A. To automatically import data from external systems into Kafka topics

B. To export data from Kafka topics to external systems

C. To monitor Kafka cluster health

D. To manage Kafka topic partitions

Solution

1. Step 1: Understand the role of source connectors

   Source connectors are designed to bring data from outside systems into Kafka topics automatically.

2. Step 2: Differentiate from sink connectors

   Sink connectors do the opposite by exporting data from Kafka to external systems, so they are not the correct answer here.

3. Final Answer:

   To automatically import data from external systems into Kafka topics -> Option A

4. Quick Check:

   Source connector = import data [OK]

Hint: Source connectors bring data into Kafka topics [OK]

Common Mistakes:

• Confusing source connectors with sink connectors
• Thinking connectors manage Kafka internals like partitions
• Assuming connectors monitor Kafka health

2. Which of the following is the correct JSON field to specify the connector class in a Kafka source connector configuration?

easy

A. "sourceConnector"

B. "source.class"

C. "connectorType"

D. "connector.class"

Solution

1. Step 1: Recall Kafka Connect configuration syntax

   The key to specify the connector implementation class is always "connector.class" in the JSON config.

2. Step 2: Eliminate incorrect options

   Other options like "source.class", "connectorType", or "sourceConnector" are not valid Kafka Connect configuration keys.

3. Final Answer:

   "connector.class" -> Option D

4. Quick Check:

   Connector class key = "connector.class" [OK]

Hint: Use "connector.class" to specify connector type [OK]

Common Mistakes:

• Using incorrect or made-up JSON keys
• Confusing source connector config keys with sink connector keys
• Omitting quotes around JSON keys

3. Given this Kafka Connect source connector configuration snippet:

{
  "name": "my-source-connector",
  "connector.class": "FileStreamSource",
  "tasks.max": "1",
  "file": "/tmp/input.txt",
  "topic": "test-topic"
}

What will happen when this connector runs?

medium

A. It fails because 'tasks.max' must be an integer, not a string

B. It reads data from /tmp/input.txt and writes it to the Kafka topic 'test-topic'

C. It creates a new Kafka topic named '/tmp/input.txt'

D. It writes data from Kafka topic 'test-topic' to /tmp/input.txt

Solution

1. Step 1: Identify connector type and purpose

   The "FileStreamSource" connector reads data from a file and sends it to a Kafka topic.

2. Step 2: Understand configuration keys

   "file" specifies the input file, "topic" specifies the Kafka topic to write to, and "tasks.max" as a string is acceptable in JSON configs.

3. Final Answer:

   It reads data from /tmp/input.txt and writes it to the Kafka topic 'test-topic' -> Option B

4. Quick Check:

   FileStreamSource reads file to topic [OK]

Hint: FileStreamSource reads file into Kafka topic [OK]

Common Mistakes:

• Confusing source and sink connectors
• Thinking 'tasks.max' must be integer type (string is valid in JSON)
• Assuming connector creates topics automatically

4. You configured a source connector with this JSON:

{
  "name": "my-connector",
  "connector.class": "JdbcSourceConnector",
  "tasks.max": 2,
  "connection.url": "jdbc:mysql://localhost:3306/mydb",
  "topic.prefix": "db-"
}

But the connector fails to start. What is the most likely cause?

medium

A. Missing required property 'mode' for JdbcSourceConnector

B. Incorrect 'tasks.max' value; must be a string

C. The 'topic.prefix' should be 'topic.name' instead

D. The 'connection.url' must use PostgreSQL syntax

Solution

1. Step 1: Review JdbcSourceConnector required properties

   JdbcSourceConnector requires a 'mode' property to specify how to query the database (e.g., 'incrementing', 'timestamp+incrementing').

2. Step 2: Check other properties

   'tasks.max' as integer is valid, 'topic.prefix' is correct, and 'connection.url' can be MySQL syntax if using MySQL.

3. Final Answer:

   Missing required property 'mode' for JdbcSourceConnector -> Option A

4. Quick Check:

   JdbcSourceConnector needs 'mode' property [OK]

Hint: JdbcSourceConnector requires 'mode' property [OK]

Common Mistakes:

• Assuming 'tasks.max' must be string
• Confusing 'topic.prefix' with 'topic.name'
• Thinking connection URL must be PostgreSQL always

5. You want to ingest data from multiple CSV files located in a directory into Kafka using a source connector. Which approach best applies?

hard

A. Use JdbcSourceConnector with CSV files as database tables

B. Use the FileStreamSource connector with the directory path as 'file' property

C. Use a FilePulse source connector configured with the directory path to read all CSV files

D. Manually write a Kafka producer to read files and send data

Solution

1. Step 1: Identify connector capabilities

   FilePulse source connector supports reading multiple files from a directory and processing CSV files.

2. Step 2: Evaluate other options

   FileStreamSource reads only a single file, JdbcSourceConnector is for databases, and manual coding is not using source connectors.

3. Final Answer:

   Use a FilePulse source connector configured with the directory path to read all CSV files -> Option C

4. Quick Check:

   FilePulse reads multiple files from directory [OK]

Hint: FilePulse connector handles multiple files ingestion [OK]

Common Mistakes:

• Using FileStreamSource for multiple files
• Trying to use JdbcSourceConnector for CSV files
• Ignoring built-in connectors and coding manually