Kafka installation and setup - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Apache Kafka?

Apache Kafka is a distributed event streaming platform used to build real-time data pipelines and streaming applications. It allows you to publish and subscribe to streams of records in a fault-tolerant way.

beginner

Name the two main components you need to install to run Kafka locally.

You need to install Apache Kafka itself and Apache Zookeeper. Zookeeper helps Kafka manage the cluster metadata and coordination.

intermediate

What is the role of Zookeeper in Kafka setup?

Zookeeper manages and coordinates Kafka brokers. It keeps track of the status of Kafka nodes and topics, helping with leader election and configuration management.

beginner

Which command starts the Kafka server after installation?

You start the Kafka server using the command: bin/kafka-server-start.sh config/server.properties on Linux/Mac or bin\windows\kafka-server-start.bat config\server.properties on Windows.

beginner

Why is it important to start Zookeeper before Kafka?

Kafka depends on Zookeeper for managing cluster metadata. If Zookeeper is not running, Kafka cannot start properly because it won't be able to coordinate brokers and topics.

What is the first step in setting up Kafka locally?

A Start Kafka server

B Create Kafka topics

C Write a Kafka producer

D Install and start Zookeeper

Zookeeper must be installed and started first because Kafka depends on it for coordination.

Which file contains the default Kafka server configuration?

A config.json

B server.properties

C kafka.properties

D zookeeper.properties

The server.properties file holds Kafka server settings.

How do you start Zookeeper after installation?

A bin/zookeeper-server-start.sh config/zookeeper.properties

B bin/kafka-server-start.sh config/server.properties

C bin/start-zookeeper.sh

D bin/zookeeper-start.sh

The command to start Zookeeper uses the zookeeper-server-start.sh script with its config file.

What is the default port Kafka listens on?

A 9092

B 2181

C 8080

D 9090

Kafka brokers listen on port 9092 by default.

Which of these is NOT required to run Kafka locally?

A Kafka binaries

B Apache Zookeeper

C Docker

D Java Runtime Environment (JRE)

Docker is optional; Kafka can run locally without Docker if Java, Zookeeper, and Kafka binaries are installed.

Describe the steps to install and start Kafka on your local machine.

Think about the order of starting services and what software is needed.

Explain why Zookeeper is important in Kafka setup and what happens if it is not running.

Consider Kafka's dependency on Zookeeper for coordination.

Practice

1. What is the first step to start Kafka on your local machine?

easy

A. Start Zookeeper service

B. Create Kafka topics

C. Run Kafka producer

D. Install Java

Solution

1. Step 1: Understand Kafka dependencies

   Kafka requires Zookeeper to coordinate its services.

2. Step 2: Start Zookeeper before Kafka

   You must start Zookeeper first to allow Kafka to connect and run properly.

3. Final Answer:

   Start Zookeeper service -> Option A

4. Quick Check:

   Kafka needs Zookeeper first = Start Zookeeper service [OK]

Hint: Always start Zookeeper before Kafka server [OK]

Common Mistakes:

• Starting Kafka before Zookeeper
• Skipping Zookeeper start
• Trying to create topics before server runs

2. Which command correctly starts the Kafka server after Zookeeper is running?

easy

A. bin/kafka-topics.sh --create --topic test

B. bin/zookeeper-server-start.sh config/zookeeper.properties

C. bin/kafka-server-start.sh config/server.properties

D. bin/kafka-console-producer.sh --topic test

Solution

1. Step 1: Identify Kafka server start command

   The Kafka server is started using kafka-server-start.sh with server properties.

2. Step 2: Differentiate from other commands

   Zookeeper start uses zookeeper-server-start.sh; topic creation and producer commands are different tools.

3. Final Answer:

   bin/kafka-server-start.sh config/server.properties -> Option C

4. Quick Check:

   Kafka server start = bin/kafka-server-start.sh config/server.properties [OK]

Hint: Kafka server uses kafka-server-start.sh script [OK]

Common Mistakes:

• Using zookeeper start command for Kafka
• Confusing topic creation with server start
• Running producer before server

3. What will be the output when you run this command after starting Kafka server?
bin/kafka-topics.sh --list --bootstrap-server localhost:9092

medium

A. Lists all existing Kafka topics

B. Starts the Kafka server

C. Creates a new topic named 'localhost'

D. Shows Zookeeper status

Solution

1. Step 1: Understand the command purpose

   The command lists topics using the Kafka bootstrap server address.

2. Step 2: Identify output type

   It outputs all existing topics in Kafka, not starting server or creating topics.

3. Final Answer:

   Lists all existing Kafka topics -> Option A

4. Quick Check:

   --list shows topics = Lists all existing Kafka topics [OK]

Hint: Use --list with --bootstrap-server to see topics [OK]

Common Mistakes:

• Thinking it starts Kafka server
• Assuming it creates topics
• Confusing with Zookeeper commands

4. You tried to start Kafka server but got an error saying 'Could not connect to Zookeeper'. What is the likely cause?

medium

A. Kafka server is already running

B. Zookeeper service is not running

C. Topic name is missing

D. Java is not installed

Solution

1. Step 1: Analyze error message

   'Could not connect to Zookeeper' means Kafka cannot reach Zookeeper service.

2. Step 2: Identify cause

   This usually happens if Zookeeper is not started or crashed.

3. Final Answer:

   Zookeeper service is not running -> Option B

4. Quick Check:

   Connection error = Zookeeper not running [OK]

Hint: Check Zookeeper is running before Kafka start [OK]

Common Mistakes:

• Ignoring Zookeeper status
• Assuming Kafka server conflict
• Blaming missing topic name

5. You want to test Kafka by sending messages to a topic named 'test'. Which sequence of commands is correct after starting Zookeeper and Kafka server?

hard

A. Start kafka-console-consumer.sh, then create topic with kafka-topics.sh

B. Only start Zookeeper and produce messages directly

C. Run kafka-server-start.sh twice, then produce messages

D. Create topic with kafka-topics.sh, then produce messages with kafka-console-producer.sh

Solution

1. Step 1: Create the topic first

   You must create the topic 'test' before sending messages to it.

2. Step 2: Use producer to send messages

   After topic creation, use kafka-console-producer.sh to send messages to 'test'.

3. Final Answer:

   Create topic with kafka-topics.sh, then produce messages with kafka-console-producer.sh -> Option D

4. Quick Check:

   Create topic then produce = Create topic with kafka-topics.sh, then produce messages with kafka-console-producer.sh [OK]

Hint: Create topic before producing messages [OK]

Common Mistakes:

• Producing messages before topic creation
• Starting Kafka server twice unnecessarily
• Skipping Kafka server start