Database backend optimization in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Airflow's Database Backend Role

What is the primary role of the database backend in Apache Airflow?

A To store metadata about DAG runs, task instances, and scheduling information

B To execute the tasks defined in DAGs

C To serve the Airflow web UI pages to users

D To manage the worker nodes and their resource allocation

💡 Hint

Think about where Airflow keeps track of what tasks ran and when.

The database backend in Airflow stores metadata such as DAG runs, task instances, and scheduling info. It does not execute tasks or serve the UI.

💻 Command Output

intermediate

Checking Airflow Database Connection Status

What is the output of the command airflow db check when the database connection is healthy?

Apache Airflow

airflow db check

A Database connection successful

B Database connection failed: timeout error

C Database schema version mismatch

D Syntax error: invalid command

💡 Hint

Look for a positive confirmation message.

The airflow db check command outputs 'Database connection successful' if the connection is healthy.

❓ Configuration

advanced

Optimizing Airflow Database Performance

Which configuration change in airflow.cfg helps reduce database load by limiting the number of task instances fetched per query?

A set <code>webserver.workers = 10</code>

B set <code>core.parallelism = 1000</code>

C set <code>scheduler.max_tis_per_query = 100</code>

D set <code>scheduler.heartbeat_interval = 5</code>

💡 Hint

Focus on scheduler settings that control database queries.

The scheduler.max_tis_per_query setting limits how many task instances the scheduler fetches per database query, reducing load.

❓ Troubleshoot

advanced

Diagnosing Slow Airflow Scheduler Due to Database

The Airflow scheduler is running slowly. Which database-related issue is most likely causing this?

A Incorrect DAG file permissions

B Insufficient worker nodes to run tasks

C Webserver port conflict causing UI delays

D Excessive number of rows in the task_instance table without cleanup

💡 Hint

Think about what happens if the database grows too large.

If the task_instance table grows too large without cleanup, queries slow down, causing scheduler delays.

✅ Best Practice

expert

Best Practice for Airflow Database Maintenance

Which practice is best to keep the Airflow metadata database optimized and prevent performance degradation?

A Increase the number of scheduler threads without limit

B Regularly run <code>airflow db cleanup</code> or use the built-in database cleanup DAG

C Disable database connection pooling to avoid stale connections

D Store all logs in the database instead of external storage

💡 Hint

Think about how to keep the database size manageable.

Regular cleanup of old metadata using airflow db cleanup or the cleanup DAG prevents database bloat and keeps performance steady.