High availability configuration in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of enabling high availability in Airflow?

Choose the best explanation for why high availability (HA) is configured in an Airflow environment.

A To reduce the number of DAGs that can be executed simultaneously.

B To ensure the Airflow scheduler and webserver continue running without interruption during failures.

C To limit access to the Airflow UI to only one user at a time.

D To disable task retries and speed up execution.

💡 Hint

Think about what happens if a server running Airflow components crashes.

High availability ensures that critical Airflow components like the scheduler and webserver keep running even if one instance fails, preventing downtime.

💻 Command Output

intermediate

Output of checking Airflow scheduler status in HA mode

You run the command airflow scheduler --status on a multi-scheduler HA setup. What output indicates that the schedulers are running correctly in HA mode?

A Scheduler is running with leader election enabled.

B Scheduler is running but no DAGs are loaded.

C Scheduler failed to start due to port conflict.

D Scheduler is running without leader election.

💡 Hint

In HA mode, schedulers coordinate to avoid duplicate task execution.

In HA mode, Airflow schedulers use leader election to decide which scheduler runs tasks. Seeing 'leader election enabled' means HA is active and working.

❓ Configuration

advanced

Correct configuration for Airflow high availability scheduler

Which configuration snippet correctly enables high availability for the Airflow scheduler in airflow.cfg?

A

[core]
scheduler_ha_enabled = true
heartbeat_sec = 5

B

[scheduler]
enable_ha = yes
heartbeat_interval = 10

C

[scheduler]
scheduler_ha = enabled
heartbeat = 5

D

[scheduler]
enable_scheduler_ha = True
scheduler_ha_heartbeat_sec = 5

💡 Hint

Check the exact parameter names under the scheduler section.

The correct parameters to enable HA in Airflow scheduler are enable_scheduler_ha and scheduler_ha_heartbeat_sec under the [scheduler] section.

❓ Troubleshoot

advanced

Troubleshooting Airflow HA scheduler leader election failure

In an Airflow HA setup, the schedulers fail to elect a leader and no tasks run. Which is the most likely cause?

A The webserver port is blocked by firewall.

B The DAG files are missing from the scheduler nodes.

C The metadata database is unreachable or misconfigured.

D The Airflow UI authentication is disabled.

💡 Hint

Leader election depends on shared state in the metadata database.

Leader election requires schedulers to coordinate via the metadata database. If the database is unreachable, leader election fails and no scheduler runs tasks.

🔀 Workflow

expert

Order the steps to enable Airflow high availability for schedulers

Arrange the steps in the correct order to enable high availability for Airflow schedulers.

A 1,2,3,4

B 2,1,3,4

C 1,3,2,4

D 3,1,2,4

💡 Hint

Think about preparing shared resources before enabling features and deploying instances.

First, the metadata database must be configured for shared access. Then enable HA in config, deploy schedulers, and finally verify leader election.