Why monitoring prevents silent pipeline failures in Apache Airflow - Test Your Understanding

Practice - 5 Tasks

1fill in blank

easy

Complete the code to set the task failure callback in Airflow.

Apache Airflow

task = PythonOperator(task_id='my_task', python_callable=my_func, on_failure_callback=[1])

Drag options to blanks, or click blank then click option'

Asend_alert

Bsend_email

Clog_error

Dretry_task

Attempts:

3 left

💡 Hint

Common Mistakes

Using a function that retries the task instead of notifying.

The on_failure_callback parameter is used to specify a function like send_email that triggers when the task fails, helping to monitor failures.

2fill in blank

medium

Complete the code to add a sensor that waits for a file before running the task.

Apache Airflow

wait_for_file = [1](task_id='wait_for_file', filepath='/data/input.csv')

Drag options to blanks, or click blank then click option'

AHttpSensor

BTimeSensor

CFileSensor

DEmailSensor

Attempts:

3 left

💡 Hint

Common Mistakes

Using a sensor that waits for time or HTTP instead of a file.

FileSensor waits for a file to appear before the downstream task runs, preventing silent failures due to missing input.

3fill in blank

hard

Fix the error in the DAG definition to enable email alerts on failure.

Apache Airflow

default_args = {'owner': 'airflow', 'email': ['[e-mail]'], 'email_on_failure': [1]

dag = DAG('my_dag', default_args=default_args, schedule_interval='@daily')

Drag options to blanks, or click blank then click option'

AFalse

BNone

C0

DTrue

Attempts:

3 left

💡 Hint

Common Mistakes

Setting it to False or None disables email alerts.

Setting email_on_failure to True enables sending email alerts when a task fails.

4fill in blank

hard

Fill both blanks to create a task that retries 3 times with a 5-minute delay.

Apache Airflow

task = PythonOperator(task_id='retry_task', python_callable=my_func, retries=[1], retry_delay=timedelta(minutes=[2]))

Drag options to blanks, or click blank then click option'

A3

B5

C1

D10

Attempts:

3 left

💡 Hint

Common Mistakes

Using too few retries or zero delay.

Setting retries=3 and retry_delay=timedelta(minutes=5) makes the task retry 3 times with 5 minutes between attempts, helping catch transient failures.

5fill in blank

hard

Fill all three blanks to create a DAG with a start date, schedule, and catchup disabled.

Apache Airflow

dag = DAG('example_dag', start_date=datetime([1], 1, 1), schedule_interval='[2]', catchup=[3])

Drag options to blanks, or click blank then click option'

A2024

B@daily

CFalse

DTrue

Attempts:

3 left

💡 Hint

Common Mistakes

Setting catchup to True causes backfill runs.

Setting start_date=datetime(2024, 1, 1), schedule_interval='@daily', and catchup=False creates a DAG that runs daily from 2024 without backfilling missed runs, preventing silent failures from unexpected catchups.