Key broker metrics in Kafka - Time & Space Complexity

Time Complexity: Key broker metrics

O(n)

Understanding Time Complexity

When working with Kafka brokers, it's important to understand how key metrics affect performance.

We want to see how the time to process messages grows as load increases.

Scenario Under Consideration

Analyze the time complexity of the following Kafka broker metric collection snippet.

    // Pseudocode for broker metric collection
    for (topicPartition in broker.topicPartitions) {
      val messages = broker.fetchMessages(topicPartition)
      metrics.record(messages.count())
    }
    

This code collects message counts for each topic partition on the broker.

Identify Repeating Operations

Look for repeated actions that take time.

• Primary operation: Looping over all topic partitions on the broker.
• How many times: Once per topic partition, which depends on the number of partitions.

How Execution Grows With Input

As the number of topic partitions grows, the work grows too.

Input Size (topic partitions)	Approx. Operations
10	10 metric recordings
100	100 metric recordings
1000	1000 metric recordings

Pattern observation: The work grows directly with the number of topic partitions.

Final Time Complexity

Time Complexity: O(n)

This means the time to collect metrics grows linearly with the number of topic partitions.

Common Mistake

[X] Wrong: "Collecting metrics is always constant time regardless of partitions."

[OK] Correct: Each partition requires separate metric collection, so more partitions mean more work.

Interview Connect

Understanding how broker metrics scale helps you design systems that stay responsive as load grows.

Self-Check

"What if we aggregated metrics across all partitions in one step? How would the time complexity change?"

Practice

1.

What does the MessagesInPerSec metric in Kafka broker monitoring indicate?

easy

A. The number of messages the broker receives per second

B. The number of active connections to the broker

C. The total disk space used by the broker

D. The number of partitions in the broker

Solution

1. Step 1: Understand the metric name

   The metric MessagesInPerSec literally means messages coming in per second to the broker.

2. Step 2: Match the metric to its meaning

   This metric counts how many messages the broker receives every second, showing its input load.

3. Final Answer:

   The number of messages the broker receives per second -> Option A

4. Quick Check:

   MessagesInPerSec = messages received per second [OK]

Hint: Look for 'In' to find incoming message rate [OK]

Common Mistakes:

• Confusing message count with connection count
• Thinking it measures disk usage
• Mixing it up with partition count

2.

Which of the following is the correct JMX metric name to monitor the broker's outgoing bytes per second?

easy

A. MessagesOutPerSec

B. BytesInPerSec

C. BytesOutPerSec

D. BytesTotalPerSec

Solution

1. Step 1: Identify the direction of data flow

   Outgoing bytes means data sent out from the broker, so the metric should say 'Out'.

2. Step 2: Match the exact metric name

   The correct metric for outgoing bytes per second is BytesOutPerSec.

3. Final Answer:

   BytesOutPerSec -> Option C

4. Quick Check:

   Outgoing bytes = BytesOutPerSec [OK]

Hint: Look for 'Out' and 'Bytes' in metric name [OK]

Common Mistakes:

• Choosing BytesInPerSec which is incoming data
• Confusing messages with bytes
• Using a non-existent metric name

3.

Given the following JMX metric values for a Kafka broker:
MessagesInPerSec = 500
BytesInPerSec = 1024000
BytesOutPerSec = 2048000
What does this tell you about the broker's activity?

medium

A. The broker's incoming and outgoing bytes are equal

B. The broker is receiving more messages than it sends out

C. The broker is idle with no message traffic

D. The broker is sending out twice as much data as it receives in messages

Solution

1. Step 1: Analyze the metric values

   MessagesInPerSec is 500, BytesInPerSec is 1,024,000 bytes, BytesOutPerSec is 2,048,000 bytes.

2. Step 2: Compare incoming and outgoing bytes

   Outgoing bytes (2,048,000) is exactly twice the incoming bytes (1,024,000), meaning the broker sends out twice as much data as it receives.

3. Final Answer:

   The broker is sending out twice as much data as it receives in messages -> Option D

4. Quick Check:

   BytesOutPerSec = 2 x BytesInPerSec [OK]

Hint: Compare BytesOutPerSec and BytesInPerSec values [OK]

Common Mistakes:

• Ignoring the difference between bytes and messages
• Assuming equal incoming and outgoing bytes
• Misreading the numeric values

4.

Consider this snippet monitoring Kafka broker metrics in Java:

double bytesIn = jmxClient.getAttribute("kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec");
double bytesOut = jmxClient.getAttribute("kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec");
if (bytesIn > bytesOut) {
    System.out.println("More incoming data");
} else {
    System.out.println("More outgoing data");
}

What is the likely bug in this code?

medium

A. The code does not handle the case when bytesIn equals bytesOut

B. The comparison should use >= to handle equal values

C. The metric names are incorrect and will cause an error

D. The JMX attribute values should be cast to int, not double

Solution

1. Step 1: Review the if-else logic

   The code checks if bytesIn is greater than bytesOut, else prints "More outgoing data".

2. Step 2: Identify missing case handling

   If bytesIn equals bytesOut, the else block runs, printing "More outgoing data" which is misleading.

3. Final Answer:

   The code does not handle the case when bytesIn equals bytesOut -> Option A

4. Quick Check:

   Equal values need explicit handling [OK]

Hint: Check if equal values are handled in if-else [OK]

Common Mistakes:

• Assuming metric names are wrong
• Ignoring equal value scenario
• Casting to wrong data type without error

5.

You want to alert if the Kafka broker's RequestHandlerAvgIdlePercent metric drops below 20%, indicating high load. Which approach correctly implements this check in pseudocode?

1. Fetch RequestHandlerAvgIdlePercent metric value
2. If value < 20 then alert 'High broker load'

hard

A. if (RequestHandlerAvgIdlePercent > 20) { alert('High broker load'); }

B. if (RequestHandlerAvgIdlePercent < 20) { alert('High broker load'); }

C. if (RequestHandlerAvgIdlePercent == 20) { alert('High broker load'); }

D. if (RequestHandlerAvgIdlePercent != 20) { alert('High broker load'); }

Solution

1. Step 1: Understand the metric meaning

   RequestHandlerAvgIdlePercent shows how idle the request handler threads are; lower means busier.

2. Step 2: Apply the alert condition

   If this value is less than 20%, it means the broker is busy, so alerting on < 20 is correct.

3. Final Answer:

   if (RequestHandlerAvgIdlePercent < 20) { alert('High broker load'); } -> Option B

4. Quick Check:

   Idle percent < 20 means high load [OK]

Hint: Alert when idle percent is less than threshold [OK]

Common Mistakes:

• Using greater than instead of less than
• Checking equality only
• Using not equal which triggers too often