JMX metrics in Kafka - Commands & Configuration

Introduction

JMX metrics help you watch how your Kafka server is doing by showing numbers about its health and activity. This helps you find problems early and keep your system running smoothly.

When you want to see how many messages your Kafka broker is processing per second.

When you need to check if your Kafka server is running out of memory or CPU.

When you want to monitor the number of active connections to your Kafka broker.

When you want to track the lag of consumers to ensure they are keeping up with producers.

When you want to collect metrics to send to a monitoring system like Prometheus or Grafana.

Config File - kafka-server-start.sh

kafka-server-start.sh

#!/bin/bash

KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
JMX_PORT=9999
export KAFKA_HEAP_OPTS
export JMX_PORT

exec /usr/bin/kafka-server-start /etc/kafka/server.properties

This script sets the Java heap size for Kafka and enables JMX on port 9999. The JMX_PORT environment variable tells Kafka to open that port for JMX monitoring. The last line starts the Kafka server with its configuration file.

Commands

This command lists all Java processes running on the machine to confirm Kafka is running.

Terminal

jps

Expected Output

12345 Kafka

67890 Jps

This command opens the JMX console connected to Kafka's JMX port to view live metrics in a graphical interface.

Terminal

jconsole localhost:9999

Expected Output

No output (command runs silently)

If Kafka is configured with a JMX exporter, this command fetches metrics in text format for monitoring tools.

Terminal

curl http://localhost:9999/metrics

Expected Output

kafka_server_broker_topic_metrics_messages_in_total 123456

kafka_server_broker_topic_metrics_bytes_in_total 78901234

Key Concept

If you remember nothing else from JMX metrics, remember: JMX lets you watch Kafka's health and activity numbers live to catch problems early.

Common Mistakes

Not setting the JMX_PORT environment variable before starting Kafka.

Kafka won't open the JMX port, so no metrics will be available for monitoring.

Always export JMX_PORT with a valid port number before starting Kafka.

Trying to connect to JMX on the wrong port or host.

The connection will fail because Kafka listens on the port set by JMX_PORT on the correct host.

Verify the JMX_PORT value and connect to the correct host and port.

Expecting metrics output from curl without configuring a JMX exporter.

Kafka does not expose HTTP metrics by default; curl will fail or return nothing.

Install and configure a JMX exporter like Prometheus JMX exporter to expose metrics over HTTP.

Summary

Set the JMX_PORT environment variable before starting Kafka to enable JMX metrics.

Use jconsole or similar tools to connect to Kafka's JMX port and view metrics live.

Configure a JMX exporter to expose metrics over HTTP for automated monitoring.