JMX metrics in Kafka - Mini Project: Build & Apply

Collecting Kafka JMX Metrics

📖 Scenario: You are managing a Kafka server and want to monitor its performance. Kafka exposes many metrics through JMX (Java Management Extensions). You will write a simple setup to collect some key JMX metrics from Kafka.

🎯 Goal: Build a small program that connects to Kafka's JMX interface, selects specific metrics, and prints their values.

📋 What You'll Learn

Create a dictionary with Kafka JMX metric names and their object names

Add a variable to specify the JMX port number

Write a loop to simulate fetching metric values from JMX

Print the collected metrics in a readable format

💡 Why This Matters

🌍 Real World

Kafka administrators monitor JMX metrics to keep track of message rates, data throughput, and broker health.

💼 Career

Understanding how to collect and process Kafka JMX metrics is useful for roles in DevOps, Site Reliability Engineering, and backend system monitoring.

1

Setup Kafka JMX metrics dictionary

Create a dictionary called kafka_metrics with these exact entries: 'MessagesInPerSec': 'kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec', 'BytesInPerSec': 'kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec', and 'BytesOutPerSec': 'kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec'.

Kafka

# Create the kafka_metrics dictionary with exact entries
# Your code here

Hint

Use curly braces to create a dictionary. Each key is a metric name string, and each value is the exact JMX object name string.

2

Set the JMX port number

Create a variable called jmx_port and set it to 9999.

Kafka

kafka_metrics = {
    'MessagesInPerSec': 'kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec',
    'BytesInPerSec': 'kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec',
    'BytesOutPerSec': 'kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec'
}
# Set the JMX port number
# Your code here

Hint

Just assign the number 9999 to the variable named jmx_port.

3

Simulate fetching metric values

Write a for loop using variables metric and object_name to iterate over kafka_metrics.items(). Inside the loop, create a dictionary called collected_metrics before the loop and add each metric with a simulated value of 100.

Kafka

kafka_metrics = {
    'MessagesInPerSec': 'kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec',
    'BytesInPerSec': 'kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec',
    'BytesOutPerSec': 'kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec'
}
jmx_port = 9999

# Create collected_metrics dictionary
collected_metrics = {}

# Loop over kafka_metrics.items() and add metric with value 100
# Your code here

Hint

Initialize collected_metrics as an empty dictionary. Use a for loop with metric and object_name to add each metric with value 100.

4

Print the collected metrics

Write a print statement to display the collected_metrics dictionary.

Kafka

kafka_metrics = {
    'MessagesInPerSec': 'kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec',
    'BytesInPerSec': 'kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec',
    'BytesOutPerSec': 'kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec'
}
jmx_port = 9999

collected_metrics = {}
for metric, object_name in kafka_metrics.items():
    collected_metrics[metric] = 100

# Print the collected_metrics dictionary
# Your code here

Hint

Use print(collected_metrics) to show the dictionary.

Practice

1. What is the primary purpose of JMX metrics in Kafka?

easy

A. To configure Kafka brokers dynamically

B. To send messages between Kafka topics

C. To monitor Kafka's internal performance and health

D. To store data permanently in Kafka

Solution

1. Step 1: Understand JMX metrics role

   JMX metrics provide data about Kafka's internal state and performance.

2. Step 2: Identify the correct purpose

   Monitoring performance and health is the main use of JMX metrics in Kafka.

3. Final Answer:

   To monitor Kafka's internal performance and health -> Option C

4. Quick Check:

   JMX metrics = Monitoring Kafka [OK]

Hint: JMX metrics are for monitoring, not messaging or storage [OK]

Common Mistakes:

• Confusing metrics with message sending
• Thinking JMX configures Kafka brokers
• Assuming JMX stores Kafka data

2. Which of the following is the correct way to identify a Kafka JMX metric?

easy

A. Using a topic name and partition number

B. Using a unique ObjectName and attribute

C. Using a consumer group ID

D. Using a broker ID and port number

Solution

1. Step 1: Recall JMX metric identification

   Each JMX metric is accessed by a unique ObjectName and attribute in Kafka.

2. Step 2: Compare options

   Only Using a unique ObjectName and attribute correctly describes the identification method for JMX metrics.

3. Final Answer:

   Using a unique ObjectName and attribute -> Option B

4. Quick Check:

   JMX metric ID = ObjectName + attribute [OK]

Hint: JMX metrics use ObjectName and attribute, not topic or group IDs [OK]

Common Mistakes:

• Mixing JMX metric IDs with Kafka topic or consumer IDs
• Confusing broker info with metric identifiers
• Assuming partition number identifies metrics

3. Given the following Java snippet to get a Kafka JMX metric attribute, what will be the output if the metric value is 150?

ObjectName name = new ObjectName("kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec");
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
Integer value = (Integer) mbs.getAttribute(name, "Count");
System.out.println(value);

medium

A. MBeanServerException

B. null

C. Count

D. 150

Solution

1. Step 1: Understand the code's purpose

   The code queries the JMX MBeanServer for the 'Count' attribute of the specified metric.

2. Step 2: Interpret the output

   If the metric value is 150, the printed output will be the integer 150.

3. Final Answer:

   150 -> Option D

4. Quick Check:

   Metric value printed = 150 [OK]

Hint: JMX attribute value prints directly if metric exists [OK]

Common Mistakes:

• Expecting attribute name instead of value
• Assuming null if metric exists
• Confusing exceptions with normal output

4. You try to read a Kafka JMX metric but get a javax.management.InstanceNotFoundException. What is the most likely cause?

medium

A. The ObjectName used does not match any registered MBean

B. The Kafka broker is down

C. The attribute name is misspelled

D. The MBeanServer is not initialized

Solution

1. Step 1: Understand the exception meaning

   InstanceNotFoundException means the requested MBean ObjectName is not found in the MBeanServer.

2. Step 2: Identify the cause

   If the ObjectName does not match any registered MBean, this exception occurs.

3. Final Answer:

   The ObjectName used does not match any registered MBean -> Option A

4. Quick Check:

   InstanceNotFoundException = Wrong ObjectName [OK]

Hint: Check ObjectName spelling if InstanceNotFoundException occurs [OK]

Common Mistakes:

• Blaming attribute name for InstanceNotFoundException
• Assuming broker down causes this exception
• Thinking MBeanServer is uninitialized

5. You want to monitor Kafka's message throughput using JMX metrics and alert if MessagesInPerSec.Count exceeds 1000. Which approach correctly combines JMX metric reading and alert logic in Java?

hard

A. Query the MBeanServer for the metric attribute, then compare the value to 1000 and trigger alert if greater

B. Subscribe to Kafka topic and count messages manually, ignoring JMX metrics

C. Use Kafka consumer lag metrics instead of MessagesInPerSec

D. Restart Kafka broker when MessagesInPerSec.Count exceeds 1000

Solution

1. Step 1: Understand monitoring with JMX

   To monitor throughput, read the MessagesInPerSec.Count attribute from Kafka's JMX MBeanServer.

2. Step 2: Implement alert logic

   Compare the retrieved metric value to 1000 and trigger an alert if it exceeds this threshold.

3. Final Answer:

   Query the MBeanServer for the metric attribute, then compare the value to 1000 and trigger alert if greater -> Option A

4. Quick Check:

   Read metric + compare + alert = Correct approach [OK]

Hint: Read metric value then compare to threshold for alert [OK]

Common Mistakes:

• Ignoring JMX metrics and counting messages manually
• Using unrelated metrics like consumer lag
• Restarting broker instead of alerting