Standalone vs distributed mode in Kafka - Hands-On Comparison

Understanding Standalone vs Distributed Mode in Kafka

📖 Scenario: You are learning how Apache Kafka works in two main modes: standalone and distributed. Kafka is a system that helps send messages between programs quickly and reliably. Understanding these modes helps you set up Kafka correctly for small or big projects.

🎯 Goal: You will create simple Kafka configuration files for both standalone and distributed modes. Then, you will write commands to start Kafka in each mode and observe the differences.

📋 What You'll Learn

Create a Kafka configuration file for standalone mode named server-standalone.properties with specific settings.

Create a Kafka configuration file for distributed mode named server-distributed.properties with specific settings.

Write commands to start Kafka using each configuration file.

Print messages explaining which mode is running.

💡 Why This Matters

🌍 Real World

Kafka is widely used in companies to handle large streams of data. Knowing how to set it up in standalone or distributed mode helps you manage data pipelines effectively.

💼 Career

Many jobs in data engineering and backend development require understanding Kafka's deployment modes to build scalable and reliable systems.

1

Create Kafka configuration for standalone mode

Create a file named server-standalone.properties with these exact lines:
broker.id=0,
log.dirs=/tmp/kafka-logs-standalone,
zookeeper.connect=localhost:2181

Kafka

# Create server-standalone.properties file with the required settings
# Your code here

Need a hint?

These settings tell Kafka to run as a single broker using local Zookeeper.

2

Create Kafka configuration for distributed mode

Create a file named server-distributed.properties with these exact lines:
broker.id=1,
log.dirs=/tmp/kafka-logs-distributed,
zookeeper.connect=localhost:2181,
num.network.threads=3

Kafka

broker.id=0
log.dirs=/tmp/kafka-logs-standalone
zookeeper.connect=localhost:2181
# Create server-distributed.properties file with the required settings
# Your code here

Need a hint?

These settings prepare Kafka to run as part of a cluster with multiple brokers.

3

Write commands to start Kafka in both modes

Write two commands:
1. bin/kafka-server-start.sh config/server-standalone.properties & to start standalone mode.
2. bin/kafka-server-start.sh config/server-distributed.properties & to start distributed mode.

Kafka

broker.id=1
log.dirs=/tmp/kafka-logs-distributed
zookeeper.connect=localhost:2181
num.network.threads=3

# Write commands to start Kafka in standalone and distributed modes
# Your code here

Need a hint?

Use the kafka-server-start.sh script with the correct config file and run in background.

4

Print messages explaining the running mode

Write two print statements:
1. print("Kafka is running in standalone mode")
2. print("Kafka is running in distributed mode")

Kafka

broker.id=1
log.dirs=/tmp/kafka-logs-distributed
zookeeper.connect=localhost:2181
num.network.threads=3

# Start Kafka in standalone mode
bin/kafka-server-start.sh config/server-standalone.properties &

# Start Kafka in distributed mode
bin/kafka-server-start.sh config/server-distributed.properties &

# Print messages about running modes
# Your code here

Need a hint?

Use print statements exactly as shown to display the mode messages.