Standalone Mode Kafka Connect: What It Is and How It Works

Standalone mode in Kafka Connect is a simple way to run connectors on a single machine without distributed coordination. It is ideal for development, testing, or small setups where you don't need fault tolerance or scalability. In this mode, Kafka Connect runs as a single process managing connectors and tasks locally.

How It Works

Standalone mode in Kafka Connect works like running a single worker that handles all the connector tasks on one machine. Imagine it as a solo chef in a kitchen who prepares all dishes alone without help. This means there is no sharing of work or coordination with other workers.

Because it runs on one machine, it is simple to set up and manage. The configuration and connector tasks are stored locally, so if the process stops, all connectors stop too. This mode does not provide automatic recovery or load balancing like distributed mode.

Standalone mode is great for quick testing or small projects where you want to move data between Kafka and other systems without the complexity of a cluster.

Example

This example shows a simple standalone worker configuration and a connector properties file to run a file source connector that reads data from a local file and sends it to a Kafka topic.

bash

# worker.properties
bootstrap.servers=localhost:9092
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
offset.storage.file.filename=/tmp/connect.offsets

# connector-file-source.properties
name=local-file-source
connector.class=FileStreamSourceConnector
tasks.max=1
file=/tmp/input.txt
topic=test-topic

# Command to start Kafka Connect in standalone mode
connect-standalone.sh worker.properties connector-file-source.properties

Output

INFO Kafka Connect started in standalone mode
INFO Connector local-file-source started
INFO Reading from /tmp/input.txt and sending to topic test-topic

When to Use

Use standalone mode when you want a simple, quick setup without the need for multiple machines or fault tolerance. It is perfect for:

• Development and testing environments
• Small projects or proof of concepts
• Single-node deployments where high availability is not required

For production systems needing scalability and fault tolerance, distributed mode is recommended instead.

Key Points

• Standalone mode runs Kafka Connect as a single process on one machine.
• It does not support fault tolerance or scaling.
• Configuration and offsets are stored locally.
• Ideal for development, testing, and small setups.
• Use distributed mode for production and multi-node clusters.

Key Takeaways

Standalone mode runs Kafka Connect on a single machine without coordination.

It is simple to set up and best for development or small projects.

No fault tolerance or scalability is provided in standalone mode.

Use distributed mode for production environments needing reliability.

Configuration and offsets are stored locally in standalone mode.