How to Use Prometheus for Microservices Monitoring

Use Prometheus by exposing metrics endpoints in each microservice, configuring Prometheus to scrape these endpoints regularly, and then querying or alerting on collected data. This setup helps monitor the health and performance of microservices efficiently.

Syntax

Prometheus works by scraping HTTP endpoints that expose metrics in a specific format. Each microservice must expose a /metrics endpoint. Prometheus configuration defines which endpoints to scrape and how often.

Key parts:

• scrape_configs: List of targets (microservices) to scrape.
• metrics_path: URL path where metrics are exposed, usually /metrics.
• job_name: Logical name for the group of targets.
• static_configs: Static list of target addresses.

yaml

scrape_configs:
  - job_name: 'microservice-a'
    metrics_path: '/metrics'
    static_configs:
      - targets: ['localhost:8080']

Example

This example shows a simple microservice in Go exposing Prometheus metrics and a Prometheus config scraping it.

The microservice exposes a counter metric incremented on each request.

go

package main

import (
	"net/http"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promhttp"
)

var (
	requestCount = prometheus.NewCounter(
		prometheus.CounterOpts{
			Name: "http_requests_total",
			Help: "Total number of HTTP requests",
		},
	)
)

func main() {
	prometheus.MustRegister(requestCount)

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		requestCount.Inc()
		w.Write([]byte("Hello from microservice!"))
	})

	http.Handle("/metrics", promhttp.Handler())

	http.ListenAndServe(":8080", nil)
}

Output

When running, visiting http://localhost:8080/ increments the counter.
Visiting http://localhost:8080/metrics shows metrics like:
# HELP http_requests_total Total number of HTTP requests
# TYPE http_requests_total counter
http_requests_total 5

Common Pitfalls

• Not exposing a /metrics endpoint or exposing it incorrectly.
• Forgetting to register custom metrics in the microservice code.
• Misconfiguring Prometheus targets, causing no data to be scraped.
• Scraping too frequently or too rarely, leading to performance issues or stale data.
• Not securing metrics endpoints in production, exposing sensitive data.

go

Wrong way:
http.HandleFunc("/metrics", func(w http.ResponseWriter, r *http.Request) {
	w.Write([]byte("not prometheus format"))
})

Right way:
import "github.com/prometheus/client_golang/prometheus/promhttp"
http.Handle("/metrics", promhttp.Handler())

Quick Reference

Concept	Description
Metrics Endpoint	HTTP path (/metrics) exposing Prometheus metrics
Prometheus Scrape	Prometheus polls endpoints at intervals to collect data
Job Name	Logical grouping of targets in Prometheus config
Custom Metrics	User-defined counters, gauges, histograms in microservices
Alerting	Rules in Prometheus to notify on metric thresholds

Key Takeaways

Expose a /metrics HTTP endpoint in each microservice for Prometheus to scrape.

Configure Prometheus with scrape_configs listing all microservice endpoints.

Register and update custom metrics properly in your microservice code.

Avoid common mistakes like missing endpoints or wrong metric formats.

Use Prometheus queries and alerting to monitor microservice health effectively.