Apache Airflow for ML orchestration in MLOps - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Apache Airflow used for in ML projects?

Apache Airflow helps automate and schedule tasks in machine learning workflows, like data preparation, model training, and deployment.

beginner

What is a DAG in Apache Airflow?

A DAG (Directed Acyclic Graph) is a set of tasks with dependencies that Airflow runs in order, representing the workflow steps.

intermediate

How does Airflow ensure tasks run in the correct order?

Airflow uses dependencies defined in the DAG to run tasks only after their upstream tasks have completed successfully.

beginner

Name two common operators used in Airflow for ML workflows.

PythonOperator to run Python code and BashOperator to run shell commands are common operators in ML workflows.

intermediate

Why is Airflow useful for retraining ML models regularly?

Airflow can schedule retraining tasks automatically at set intervals, ensuring models stay updated without manual work.

What does DAG stand for in Apache Airflow?

A Dynamic Application Gateway

B Data Analysis Group

C Distributed Automation Grid

D Directed Acyclic Graph

DAG means Directed Acyclic Graph, which defines the order of tasks in Airflow.

Which Airflow component defines the workflow steps and their order?

A Operator

B DAG

C Task Instance

D Scheduler

The DAG defines the workflow steps and their dependencies.

Which operator would you use to run a Python function in Airflow?

A PythonOperator

B BashOperator

C EmailOperator

D DockerOperator

PythonOperator runs Python code inside Airflow tasks.

How does Airflow handle task failures by default?

A It stops the entire workflow immediately

B It ignores failures and continues

C It retries the task a set number of times

D It sends an email but continues

Airflow retries failed tasks based on retry settings before marking them as failed.

What is a key benefit of using Airflow for ML model retraining?

A Scheduling retraining automatically

B Manual triggering of retraining

C Replacing the need for data scientists

D Automatically improving model accuracy

Airflow schedules retraining tasks automatically to keep models updated.

Explain how Apache Airflow helps manage machine learning workflows.

Think about how Airflow organizes and runs steps like data prep and model training.

Describe the role of operators in Airflow and name two used in ML pipelines.

Operators are like tools Airflow uses to do work.