Kafka Connect architecture - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Kafka Connect?

Kafka Connect is a tool for streaming data between Apache Kafka and other systems in a scalable and reliable way.

beginner

Name the two main types of connectors in Kafka Connect.

Source connectors (to pull data into Kafka) and Sink connectors (to push data out of Kafka).

intermediate

What role does the Kafka Connect worker play?

A Kafka Connect worker runs connectors and tasks, managing data movement and ensuring fault tolerance.

intermediate

Explain the difference between standalone and distributed mode in Kafka Connect.

Standalone mode runs a single worker for simple or development use. Distributed mode runs multiple workers for scalability and fault tolerance.

advanced

How does Kafka Connect ensure fault tolerance?

By storing connector configurations and offsets in Kafka topics, allowing workers to recover and continue processing after failures.

What type of connector would you use to bring data from a database into Kafka?

A Source connector

B Sink connector

C Worker connector

D Stream connector

Source connectors pull data from external systems into Kafka.

Which Kafka Connect mode supports running multiple workers for scalability?

A Cluster mode

B Standalone mode

C Single mode

D Distributed mode

Distributed mode allows multiple workers to run together for scalability and fault tolerance.

Where does Kafka Connect store connector configurations and offsets?

A In local files only

B In Kafka topics

C In a database

D In memory only

Kafka Connect stores configurations and offsets in Kafka topics to enable fault tolerance.

What is the main responsibility of a Kafka Connect worker?

A Running connectors and tasks

B Producing Kafka topics

C Managing Kafka brokers

D Consuming Kafka topics

Workers run connectors and their tasks to move data between Kafka and other systems.

Which connector type sends data from Kafka to an external system?

A Source connector

B Producer connector

C Sink connector

D Consumer connector

Sink connectors push data from Kafka to external systems.

Describe the architecture of Kafka Connect and how it moves data between Kafka and other systems.

Think about the roles of workers and connectors and how data flows.

Explain the difference between standalone and distributed modes in Kafka Connect and when you might use each.

Consider simple vs production environments.