Key broker metrics in Kafka - Step-by-Step Execution

Process Flow - Key broker metrics

Start Kafka Broker

↓

Collect Metrics

↓

Broker Metrics: Request Rate, Latency, Throughput

↓

Monitor Disk Usage, Network I/O, CPU

↓

Trigger Alerts if Thresholds Exceeded

↓

Adjust Broker or Topic Configurations

↓

Repeat Monitoring Cycle

This flow shows how a Kafka broker collects and monitors key metrics continuously to ensure smooth operation.

Execution Sample

Kafka

metrics = broker.get_metrics()
for metric in metrics:
    print(f"{metric.name}: {metric.value}")

This code fetches key metrics from a Kafka broker and prints each metric's name and value.

Process Table

Step	Action	Metric Name	Metric Value	Output
1	Retrieve metric from metrics list	request_rate	1500 req/s	None
2	Retrieve metric from metrics list	request_latency	5 ms	None
3	Retrieve metric from metrics list	bytes_in_per_sec	10 MB/s	None
4	Retrieve metric from metrics list	bytes_out_per_sec	9 MB/s	None
5	Retrieve metric from metrics list	disk_usage	70%	None
6	Print metric	request_rate	1500 req/s	request_rate: 1500 req/s
7	Print metric	request_latency	5 ms	request_latency: 5 ms
8	Print metric	bytes_in_per_sec	10 MB/s	bytes_in_per_sec: 10 MB/s
9	Print metric	bytes_out_per_sec	9 MB/s	bytes_out_per_sec: 9 MB/s
10	Print metric	disk_usage	70%	disk_usage: 70%
11	End of metrics list			Execution stops after all metrics printed

💡 All key broker metrics have been printed, so the loop ends.

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	Final
metrics	empty	[request_rate,...]	[request_rate,...]	[request_rate,...]	[request_rate,...]	[request_rate,...]	[request_rate,...]
metric.name	none	request_rate	request_latency	bytes_in_per_sec	bytes_out_per_sec	disk_usage	none
metric.value	none	1500 req/s	5 ms	10 MB/s	9 MB/s	70%	none

Key Moments - 2 Insights

Why do we see multiple metrics printed one after another instead of just one?

Because the code loops through all metrics returned by get_metrics(), printing each metric's name and value as shown in execution_table rows 6 to 10.

What happens if the disk_usage metric value goes above 90%?

The monitoring system would trigger an alert to warn about high disk usage, prompting action to avoid broker issues. This is implied in the concept_flow step 'Trigger Alerts if Thresholds Exceeded'.

Visual Quiz - 3 Questions

Look at the execution_table, what is the metric value printed at step 8?

A 10 MB/s

B 9 MB/s

C 5 ms

D 1500 req/s

💡 Hint

Check the 'Metric Value' column at step 8 in the execution_table.

At which step does the code print the disk_usage metric?

A Step 6

B Step 5

C Step 10

D Step 11

💡 Hint

Look for the 'Print metric' action with 'disk_usage' in the 'Metric Name' column.

If the broker had an additional metric 'cpu_usage', how would the execution_table change?

A Replace 'disk_usage' metric with 'cpu_usage' metric.

B Add a new row for 'cpu_usage' metric after step 5 and before printing steps.

C No change, metrics are fixed.

D Remove all printing steps.

💡 Hint

New metrics appear as new rows in the metrics list before printing, as shown in steps 1-5.

Concept Snapshot

Key broker metrics in Kafka include request rate, latency, throughput, disk usage, and CPU.
These metrics are collected continuously by the broker.
Monitoring them helps detect performance issues early.
Metrics are accessed via broker APIs and printed or logged.
Alerts can be triggered if metrics exceed safe thresholds.

Full Transcript

This visual execution trace shows how a Kafka broker collects and prints key metrics like request rate, latency, bytes in/out per second, and disk usage. The code calls get_metrics() to get a list of metrics, then loops through each metric to print its name and value. The execution table details each step, showing metric names and values fetched and printed. Variables track the metrics list and current metric's name and value. Key moments clarify why multiple metrics print and what happens if disk usage is high. The quiz tests understanding of metric values at specific steps and how adding a new metric affects the flow. The snapshot summarizes the importance and usage of key broker metrics for monitoring Kafka brokers.

Practice

1.

What does the MessagesInPerSec metric in Kafka broker monitoring indicate?

easy

A. The number of messages the broker receives per second

B. The number of active connections to the broker

C. The total disk space used by the broker

D. The number of partitions in the broker

Solution

1. Step 1: Understand the metric name

   The metric MessagesInPerSec literally means messages coming in per second to the broker.

2. Step 2: Match the metric to its meaning

   This metric counts how many messages the broker receives every second, showing its input load.

3. Final Answer:

   The number of messages the broker receives per second -> Option A

4. Quick Check:

   MessagesInPerSec = messages received per second [OK]

Hint: Look for 'In' to find incoming message rate [OK]

Common Mistakes:

• Confusing message count with connection count
• Thinking it measures disk usage
• Mixing it up with partition count

2.

Which of the following is the correct JMX metric name to monitor the broker's outgoing bytes per second?

easy

A. MessagesOutPerSec

B. BytesInPerSec

C. BytesOutPerSec

D. BytesTotalPerSec

Solution

1. Step 1: Identify the direction of data flow

   Outgoing bytes means data sent out from the broker, so the metric should say 'Out'.

2. Step 2: Match the exact metric name

   The correct metric for outgoing bytes per second is BytesOutPerSec.

3. Final Answer:

   BytesOutPerSec -> Option C

4. Quick Check:

   Outgoing bytes = BytesOutPerSec [OK]

Hint: Look for 'Out' and 'Bytes' in metric name [OK]

Common Mistakes:

• Choosing BytesInPerSec which is incoming data
• Confusing messages with bytes
• Using a non-existent metric name

3.

Given the following JMX metric values for a Kafka broker:
MessagesInPerSec = 500
BytesInPerSec = 1024000
BytesOutPerSec = 2048000
What does this tell you about the broker's activity?

medium

A. The broker's incoming and outgoing bytes are equal

B. The broker is receiving more messages than it sends out

C. The broker is idle with no message traffic

D. The broker is sending out twice as much data as it receives in messages

Solution

1. Step 1: Analyze the metric values

   MessagesInPerSec is 500, BytesInPerSec is 1,024,000 bytes, BytesOutPerSec is 2,048,000 bytes.

2. Step 2: Compare incoming and outgoing bytes

   Outgoing bytes (2,048,000) is exactly twice the incoming bytes (1,024,000), meaning the broker sends out twice as much data as it receives.

3. Final Answer:

   The broker is sending out twice as much data as it receives in messages -> Option D

4. Quick Check:

   BytesOutPerSec = 2 x BytesInPerSec [OK]

Hint: Compare BytesOutPerSec and BytesInPerSec values [OK]

Common Mistakes:

• Ignoring the difference between bytes and messages
• Assuming equal incoming and outgoing bytes
• Misreading the numeric values

4.

Consider this snippet monitoring Kafka broker metrics in Java:

double bytesIn = jmxClient.getAttribute("kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec");
double bytesOut = jmxClient.getAttribute("kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec");
if (bytesIn > bytesOut) {
    System.out.println("More incoming data");
} else {
    System.out.println("More outgoing data");
}

What is the likely bug in this code?

medium

A. The code does not handle the case when bytesIn equals bytesOut

B. The comparison should use >= to handle equal values

C. The metric names are incorrect and will cause an error

D. The JMX attribute values should be cast to int, not double

Solution

1. Step 1: Review the if-else logic

   The code checks if bytesIn is greater than bytesOut, else prints "More outgoing data".

2. Step 2: Identify missing case handling

   If bytesIn equals bytesOut, the else block runs, printing "More outgoing data" which is misleading.

3. Final Answer:

   The code does not handle the case when bytesIn equals bytesOut -> Option A

4. Quick Check:

   Equal values need explicit handling [OK]

Hint: Check if equal values are handled in if-else [OK]

Common Mistakes:

• Assuming metric names are wrong
• Ignoring equal value scenario
• Casting to wrong data type without error

5.

You want to alert if the Kafka broker's RequestHandlerAvgIdlePercent metric drops below 20%, indicating high load. Which approach correctly implements this check in pseudocode?

1. Fetch RequestHandlerAvgIdlePercent metric value
2. If value < 20 then alert 'High broker load'

hard

A. if (RequestHandlerAvgIdlePercent > 20) { alert('High broker load'); }

B. if (RequestHandlerAvgIdlePercent < 20) { alert('High broker load'); }

C. if (RequestHandlerAvgIdlePercent == 20) { alert('High broker load'); }

D. if (RequestHandlerAvgIdlePercent != 20) { alert('High broker load'); }

Solution

1. Step 1: Understand the metric meaning

   RequestHandlerAvgIdlePercent shows how idle the request handler threads are; lower means busier.

2. Step 2: Apply the alert condition

   If this value is less than 20%, it means the broker is busy, so alerting on < 20 is correct.

3. Final Answer:

   if (RequestHandlerAvgIdlePercent < 20) { alert('High broker load'); } -> Option B

4. Quick Check:

   Idle percent < 20 means high load [OK]

Hint: Alert when idle percent is less than threshold [OK]

Common Mistakes:

• Using greater than instead of less than
• Checking equality only
• Using not equal which triggers too often