Airflow metrics with Prometheus - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Airflow Metrics Export

Which Airflow component is responsible for exposing metrics to Prometheus?

A The Airflow Scheduler exposes metrics via a built-in Prometheus exporter.

B The Airflow Worker pushes metrics directly to Prometheus using a push gateway.

C The Airflow Webserver exposes metrics through a Prometheus endpoint.

D The Airflow Metadata Database exports metrics to Prometheus automatically.

💡 Hint

Think about which Airflow service handles HTTP requests and UI.

The Airflow Webserver exposes metrics on a Prometheus endpoint, allowing Prometheus to scrape metrics. The Scheduler and Workers do not expose metrics directly, and the Metadata Database does not export metrics.

💻 Command Output

intermediate

Prometheus Metrics Endpoint Output

What is the expected output when you curl the Airflow Webserver's Prometheus metrics endpoint /metrics?

Apache Airflow

curl http://localhost:8080/metrics

A

# HELP airflow_scheduler_heartbeat_seconds Time since last scheduler heartbeat
# TYPE airflow_scheduler_heartbeat_seconds gauge
airflow_scheduler_heartbeat_seconds 12.345

B

# HELP airflow_task_duration_seconds Duration of task execution
# TYPE airflow_task_duration_seconds histogram
airflow_task_duration_seconds_bucket{le="5"} 10
airflow_task_duration_seconds_bucket{le="10"} 15

C

HTTP 404 Not Found

D

Error: Connection refused

💡 Hint

The metrics endpoint provides Prometheus formatted metrics about Airflow components.

The Airflow Webserver's /metrics endpoint exposes metrics like scheduler heartbeat as gauges. Option A shows a valid Prometheus metric output. Option A is a 404 error which means the endpoint is missing. Option A shows task duration histogram which is not exposed by default. Option A is a connection error.

❓ Configuration

advanced

Configuring Airflow to Enable Prometheus Metrics

Which configuration change is required in airflow.cfg to enable Prometheus metrics on the Airflow Webserver?

A

[metrics]
prometheus = True
port = 9090

B

[metrics]
enabled = True
exporter = prometheus

C

[webserver]
metrics_exporter = prometheus
metrics_port = 8080

D

[metrics]
enabled = True
backend = prometheus

💡 Hint

Look for the correct section and keys related to metrics export.

The correct configuration to enable Prometheus metrics is under the [metrics] section with keys 'enabled = True' and 'exporter = prometheus'. Other options either use wrong keys or sections.

❓ Troubleshoot

advanced

Troubleshooting Missing Metrics in Prometheus

You configured Airflow to expose Prometheus metrics, but Prometheus shows no data for Airflow targets. What is the most likely cause?

A The Airflow Webserver is not running or not reachable on the metrics port.

B Prometheus is not configured to scrape the Airflow Metadata Database.

C The Airflow Scheduler is not pushing metrics to Prometheus push gateway.

D The Airflow Workers are blocking the metrics endpoint with firewall rules.

💡 Hint

Check network connectivity and service status first.

If Prometheus shows no data, the most common cause is that the Airflow Webserver is down or unreachable on the metrics port. Prometheus scrapes the Webserver, not the Metadata DB or Workers. Airflow does not push metrics to a push gateway by default.

🔀 Workflow

expert

Setting Up Full Airflow Metrics Monitoring with Prometheus and Grafana

Order the steps correctly to set up Airflow metrics monitoring with Prometheus and Grafana.

A 1,3,2,4

B 2,1,3,4

C 3,2,1,4

D 1,2,3,4

💡 Hint

Think about enabling metrics first, then scraping, then visualization setup.

First, enable metrics in Airflow and restart the Webserver. Second, configure Prometheus to scrape those metrics. Third, deploy Grafana and add Prometheus as data source. Finally, import or create dashboards to visualize the metrics.

Practice

1. What is the main purpose of integrating Prometheus with Airflow metrics?

easy

A. To schedule Airflow DAGs automatically

B. To store Airflow logs permanently

C. To monitor Airflow task and system health in real-time

D. To send emails from Airflow tasks

Solution

1. Step 1: Understand Airflow metrics role

   Airflow metrics provide data about task execution and system performance.

2. Step 2: Identify Prometheus integration purpose

   Prometheus collects and visualizes these metrics for monitoring and alerting.

3. Final Answer:

   To monitor Airflow task and system health in real-time -> Option C

4. Quick Check:

   Monitoring = B [OK]

Hint: Prometheus is for monitoring, not scheduling or logging [OK]

Common Mistakes:

• Confusing monitoring with scheduling
• Thinking Prometheus stores logs
• Assuming Prometheus sends emails

2. Which configuration option must be enabled in Airflow to expose metrics for Prometheus scraping?

easy

A. metrics_enabled = False

B. statsd_on = True

C. prometheus_exporter = False

D. logging_level = DEBUG

Solution

1. Step 1: Identify Airflow metric exposure setting

   Airflow uses StatsD to collect metrics, so enabling StatsD is required.

2. Step 2: Match correct config option

   The correct option to enable StatsD is statsd_on = True.

3. Final Answer:

   statsd_on = True -> Option B

4. Quick Check:

   Enable StatsD = A [OK]

Hint: Enable StatsD to expose metrics for Prometheus [OK]

Common Mistakes:

• Disabling metrics with false flags
• Confusing logging level with metrics
• Assuming Prometheus exporter is off by default

3. Given this Airflow config snippet:

[metrics]
statsd_on = True
statsd_host = "localhost"
statsd_port = 8125

What will Prometheus scrape from Airflow?

medium

A. Airflow task execution metrics via StatsD exporter

B. Airflow DAG definitions

C. Airflow database contents

D. Airflow log files

Solution

1. Step 1: Analyze config snippet

   The snippet enables StatsD metrics collection on localhost port 8125.

2. Step 2: Understand Prometheus scraping

   Prometheus scrapes metrics exposed by StatsD exporter, which includes task execution data.

3. Final Answer:

   Airflow task execution metrics via StatsD exporter -> Option A

4. Quick Check:

   StatsD metrics = D [OK]

Hint: StatsD config means Prometheus scrapes metrics, not logs or DB [OK]

Common Mistakes:

• Thinking Prometheus scrapes DAG code
• Assuming Prometheus reads database directly
• Confusing logs with metrics

4. You enabled StatsD in Airflow but Prometheus shows no metrics. What is a likely cause?

medium

A. Prometheus scrape config missing or incorrect

B. Airflow DAGs are paused

C. Airflow scheduler is disabled

D. Airflow logs are too large

Solution

1. Step 1: Check Prometheus scraping setup

   If Prometheus shows no metrics, it may not be scraping the StatsD exporter endpoint correctly.

2. Step 2: Rule out unrelated causes

   Paused DAGs or scheduler off do not stop metrics exposure; large logs do not affect metrics scraping.

3. Final Answer:

   Prometheus scrape config missing or incorrect -> Option A

4. Quick Check:

   Scrape config issue = C [OK]

Hint: Check Prometheus scrape config if no metrics appear [OK]

Common Mistakes:

• Blaming paused DAGs for missing metrics
• Assuming scheduler off stops metrics
• Thinking logs size affects metrics

5. You want to create an alert in Prometheus for Airflow tasks that fail more than 5 times in 10 minutes. Which Prometheus query is correct?

hard

A. count(airflow_task_failures_total) == 5

B. sum(airflow_task_success_total) > 5

C. rate(airflow_task_failures_total[5m]) < 5

D. increase(airflow_task_failures_total[10m]) > 5

Solution

1. Step 1: Understand alert condition

   The alert triggers if failures increase by more than 5 in 10 minutes.

2. Step 2: Match Prometheus query

   increase(metric[10m]) > 5 checks if failures increased over 5 in last 10 minutes.

3. Step 3: Eliminate wrong options

   sum(airflow_task_success_total) > 5 checks successes, not failures; C uses wrong time and comparison; D counts current samples, not increase.

4. Final Answer:

   increase(airflow_task_failures_total[10m]) > 5 -> Option D

5. Quick Check:

   Use increase() over 10m > 5 for failure alert [OK]

Hint: Use increase() over time window to detect failure spikes [OK]

Common Mistakes:

• Using success metric instead of failure
• Using rate() incorrectly for count threshold
• Counting samples instead of increase