Kafka Manager/UI tools - Mini Project: Build & Apply

Basic Kafka Manager Setup and Topic Listing

📖 Scenario: You are working as a Kafka administrator. You want to manage Kafka topics easily using a Kafka Manager tool. This project will guide you through setting up a simple Kafka Manager configuration and listing topics using a command-line interface.

🎯 Goal: Build a simple Kafka Manager setup script that connects to a Kafka cluster and lists all available topics.

📋 What You'll Learn

Create a configuration dictionary with Kafka cluster details

Add a variable for the Kafka Manager URL

Write a function to simulate fetching topics from the Kafka cluster

Print the list of topics retrieved from the Kafka Manager

💡 Why This Matters

🌍 Real World

Kafka Managers and UI tools help administrators easily monitor and manage Kafka clusters and topics without using complex commands.

💼 Career

Understanding how to configure and interact with Kafka Manager tools is useful for roles like Kafka administrator, DevOps engineer, and backend developer.

1

Create Kafka cluster configuration

Create a dictionary called kafka_cluster with these exact entries: 'name': 'TestCluster', 'zookeeper': 'localhost:2181', and 'kafka_broker': 'localhost:9092'.

Kafka

# Create the kafka_cluster dictionary with name, zookeeper, and kafka_broker
# Your code here

Hint

Use curly braces to create a dictionary and include the exact keys and values as shown.

2

Add Kafka Manager URL configuration

Create a variable called kafka_manager_url and set it to the string 'http://localhost:9000'.

Kafka

kafka_cluster = {
    'name': 'TestCluster',
    'zookeeper': 'localhost:2181',
    'kafka_broker': 'localhost:9092'
}
# Create kafka_manager_url variable with the URL string
# Your code here

Hint

Assign the URL string exactly as shown to the variable kafka_manager_url.

3

Write function to fetch topics

Define a function called get_topics that takes no parameters and returns the list ['topic1', 'topic2', 'topic3'].

Kafka

kafka_cluster = {
    'name': 'TestCluster',
    'zookeeper': 'localhost:2181',
    'kafka_broker': 'localhost:9092'
}
kafka_manager_url = 'http://localhost:9000'
# Define get_topics function that returns a list of topics
# Your code here

Hint

Define a function with no inputs that returns the exact list of topics.

4

Print the list of topics

Call the function get_topics() and print the result using print().

Kafka

kafka_cluster = {
    'name': 'TestCluster',
    'zookeeper': 'localhost:2181',
    'kafka_broker': 'localhost:9092'
}
kafka_manager_url = 'http://localhost:9000'

def get_topics():
    return ['topic1', 'topic2', 'topic3']
# Print the list of topics returned by get_topics()
# Your code here

Hint

Use print() to show the list returned by get_topics().

Practice

1. What is the main purpose of Kafka Manager or similar UI tools for Kafka?

easy

A. To encrypt Kafka data during transmission

B. To provide a visual interface for monitoring Kafka clusters

C. To write Kafka messages automatically

D. To replace Kafka brokers with a graphical system

Solution

1. Step 1: Understand Kafka Manager's role

   Kafka Manager provides a web interface to monitor Kafka clusters visually.

2. Step 2: Compare options

   Options A, B, and C describe functions Kafka Manager does not perform.

3. Final Answer:

   To provide a visual interface for monitoring Kafka clusters -> Option B

4. Quick Check:

   Kafka Manager = Visual monitoring [OK]

Hint: Remember: Kafka Manager shows Kafka data visually [OK]

Common Mistakes:

• Thinking Kafka Manager replaces brokers
• Confusing monitoring with data encryption
• Assuming Kafka Manager writes messages automatically

2. Which command is typically used to start Kafka Manager UI after installation?

easy

A. java -jar kafka-manager.jar

B. kafka-server-start.sh

C. kafka-manager start

D. kafka-console-producer.sh

Solution

1. Step 1: Identify Kafka Manager start command

   Kafka Manager is a Java application started with java -jar kafka-manager.jar.

2. Step 2: Eliminate other options

   Options B and D start Kafka brokers or producers, not Kafka Manager. kafka-manager start is not a valid command.

3. Final Answer:

   java -jar kafka-manager.jar -> Option A

4. Quick Check:

   Start Kafka Manager = java -jar [OK]

Hint: Kafka Manager runs as a Java jar file [OK]

Common Mistakes:

• Using Kafka broker start commands instead
• Assuming a custom 'kafka-manager start' command exists
• Confusing producer commands with UI start

3. Given Kafka Manager is running and connected to a Kafka cluster, what will you see on its dashboard?

medium

A. List of topics, brokers, and consumer groups with their status

B. Only the raw Kafka log files

C. Source code of Kafka brokers

D. Encrypted Kafka messages

Solution

1. Step 1: Understand Kafka Manager dashboard content

   The dashboard shows topics, brokers, and consumer groups with their health and status.

2. Step 2: Check other options

   Raw logs, source code, or encrypted messages are not displayed on the dashboard.

3. Final Answer:

   List of topics, brokers, and consumer groups with their status -> Option A

4. Quick Check:

   Dashboard = Topics + Brokers + Consumers [OK]

Hint: Dashboard shows cluster components and status [OK]

Common Mistakes:

• Expecting raw logs or source code in UI
• Thinking encrypted messages are visible
• Confusing Kafka Manager with log viewer

4. You started Kafka Manager but the UI does not load in the browser. Which is the most likely cause?

medium

A. Kafka Manager requires a database to start

B. Kafka Manager is not connected to any Kafka cluster

C. Kafka topics are empty

D. Kafka Manager server is not running or listening on the expected port

Solution

1. Step 1: Check Kafka Manager server status

   If the UI does not load, the server might not be running or listening on the port.

2. Step 2: Evaluate other options

   Connection to Kafka cluster or empty topics won't prevent UI loading. Kafka Manager does not require a separate database to start.

3. Final Answer:

   Kafka Manager server is not running or listening on the expected port -> Option D

4. Quick Check:

   UI not loading = Server not running [OK]

Hint: Check if Kafka Manager server is running and port is open [OK]

Common Mistakes:

• Assuming empty topics block UI
• Thinking Kafka Manager needs a database to start
• Ignoring server running status

5. You want to monitor consumer lag using Kafka Manager UI. Which steps should you follow to set this up correctly?

hard

A. Install a separate lag monitoring tool and connect it to Kafka Manager

B. Modify Kafka broker configuration to enable lag monitoring

C. Add the Kafka cluster in Kafka Manager, then navigate to consumer groups to view lag metrics

D. Restart Kafka Manager with lag monitoring enabled via command line flag

Solution

1. Step 1: Add Kafka cluster to Kafka Manager

   You must first add your Kafka cluster details in Kafka Manager to connect and monitor it.

2. Step 2: Access consumer groups section

   Once connected, navigate to consumer groups in the UI to see lag metrics and consumer status.

3. Final Answer:

   Add the Kafka cluster in Kafka Manager, then navigate to consumer groups to view lag metrics -> Option C

4. Quick Check:

   Monitor lag = Add cluster + check consumer groups [OK]

Hint: Add cluster first, then check consumer groups for lag [OK]

Common Mistakes:

• Thinking lag monitoring needs separate tools
• Trying to enable lag via broker config
• Restarting Kafka Manager with flags instead of UI steps