Airflow architecture (scheduler, webserver, executor, metadata DB) - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the role of the Airflow Scheduler?

The Scheduler monitors DAG definitions and triggers tasks to run at the right time. It decides when and what tasks should be executed.

beginner

What does the Airflow Webserver do?

The Webserver provides a user interface to view DAGs, task status, logs, and manage workflows easily through a browser.

beginner

Explain the purpose of the Airflow Executor.

The Executor runs the tasks assigned by the Scheduler. It manages how and where tasks run, such as locally or on a cluster.

beginner

What is stored in the Airflow Metadata Database?

The Metadata Database stores information about DAGs, task status, schedules, and logs. It keeps track of workflow history and state.

intermediate

How do the Scheduler, Webserver, Executor, and Metadata DB work together in Airflow?

The Scheduler decides when tasks run and tells the Executor to run them. The Executor runs tasks and updates the Metadata DB with status. The Webserver shows this info to users.

Which Airflow component is responsible for triggering tasks at scheduled times?

A Scheduler

B Webserver

C Executor

D Metadata Database

The Scheduler monitors DAGs and triggers tasks when their schedule arrives.

Where can you view the status and logs of Airflow tasks?

A Executor

B Webserver

C Scheduler

D Metadata Database

The Webserver provides a user interface to view task status and logs.

What does the Airflow Executor do?

A Displays the UI

B Stores task logs

C Schedules DAG runs

D Runs tasks assigned by the Scheduler

The Executor runs the tasks that the Scheduler tells it to run.

What kind of data is stored in the Airflow Metadata Database?

A Task execution status and DAG info

B Task execution logs only

C User interface settings

D Task code files

The Metadata DB stores task status, DAG definitions, schedules, and logs.

Which component updates the Metadata Database with task status?

A Scheduler

B Webserver

C Executor

D None of the above

The Executor runs tasks and updates the Metadata DB with their status.

Describe how the Airflow Scheduler, Executor, Webserver, and Metadata Database interact to run and monitor workflows.

Think of the Scheduler as the planner, Executor as the worker, Metadata DB as the notebook, and Webserver as the dashboard.

Explain the purpose of the Airflow Metadata Database and why it is important.

It is like a diary that remembers everything about your workflows.