How to Install Kafka: Step-by-Step Guide for Beginners

To install Kafka, first download the latest Kafka binary from the official Apache website, then extract it. Start the Zookeeper service followed by the Kafka broker using the provided shell scripts.

Syntax

The basic steps to install Kafka involve these commands:

• wget [Kafka-download-URL] - downloads Kafka binaries.
• tar -xzf kafka_2.13-3.4.0.tgz - extracts the Kafka files.
• bin/zookeeper-server-start.sh config/zookeeper.properties - starts Zookeeper service.
• bin/kafka-server-start.sh config/server.properties - starts Kafka broker.

Each part is essential: downloading, extracting, and running services in order.

bash

wget https://downloads.apache.org/kafka/3.4.0/kafka_2.13-3.4.0.tgz

tar -xzf kafka_2.13-3.4.0.tgz

cd kafka_2.13-3.4.0

bin/zookeeper-server-start.sh config/zookeeper.properties

bin/kafka-server-start.sh config/server.properties

Example

This example shows how to download Kafka 3.4.0, extract it, and start Zookeeper and Kafka broker on a Linux system.

bash

wget https://downloads.apache.org/kafka/3.4.0/kafka_2.13-3.4.0.tgz

tar -xzf kafka_2.13-3.4.0.tgz

cd kafka_2.13-3.4.0

# Start Zookeeper in background
bin/zookeeper-server-start.sh -daemon config/zookeeper.properties

# Start Kafka broker in background
bin/kafka-server-start.sh -daemon config/server.properties

# Check Kafka process
ps aux | grep kafka

Output

root     12345  0.5  1.2 123456 7890 ?        Sl   10:00   0:01 /bin/java -Xmx1G -Dkafka.logs.dir=logs ... kafka.Kafka
root     12346  0.3  1.0 123456 6789 ?        Sl   10:00   0:00 /bin/java -Xmx1G -Dzookeeper.logs.dir=logs ... zookeeper.QuorumPeerMain

Common Pitfalls

Common mistakes when installing Kafka include:

• Not starting Zookeeper before Kafka broker, causing connection errors.
• Using incompatible Java versions; Kafka requires Java 8 or higher.
• Not running the start scripts with executable permissions.
• Ignoring firewall rules that block Kafka ports (default 9092).

bash

## Wrong: Starting Kafka before Zookeeper
bin/kafka-server-start.sh config/server.properties

# Error: Connection to Zookeeper failed

## Right: Start Zookeeper first
bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties

Quick Reference

Step	Command	Description
1	wget [Kafka-download-URL]	Download Kafka binaries
2	tar -xzf kafka_2.13-3.4.0.tgz	Extract Kafka files
3	bin/zookeeper-server-start.sh config/zookeeper.properties	Start Zookeeper service
4	bin/kafka-server-start.sh config/server.properties	Start Kafka broker
5	ps aux | grep kafka	Verify Kafka processes running

Key Takeaways

Always start Zookeeper before starting the Kafka broker to avoid connection errors.

Download Kafka from the official Apache website to get the latest stable version.

Ensure Java 8 or higher is installed before running Kafka.

Use the provided shell scripts to start services and run them with executable permissions.

Check running processes to confirm Kafka and Zookeeper are active.