ExternalTaskSensor for cross-DAG dependencies in Apache Airflow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of the ExternalTaskSensor in Airflow?

ExternalTaskSensor waits for a task in a different DAG to complete before allowing the current DAG to proceed. It helps manage dependencies across DAGs.

beginner

Which parameters are essential to configure an ExternalTaskSensor?

You must specify external_dag_id (the DAG to watch), external_task_id (the task to wait for), and optionally execution_date or use the default to match the current DAG's execution date.

intermediate

How does ExternalTaskSensor handle execution dates by default?

By default, ExternalTaskSensor waits for the task in the external DAG that has the same execution date as the current DAG run.

intermediate

What happens if the external task being waited on fails or never completes?

The ExternalTaskSensor will keep waiting until the task succeeds or the sensor times out or fails based on its timeout or retries settings.

advanced

Can ExternalTaskSensor be used to wait for multiple tasks or DAGs?

ExternalTaskSensor can wait for multiple tasks in the same external DAG (by providing a list to external_task_id). For multiple DAGs, use multiple sensors or custom logic.

What does ExternalTaskSensor primarily do in Airflow?

A Schedules DAG runs automatically

B Triggers a DAG run manually

C Monitors system resource usage

D Waits for a task in another DAG to complete

ExternalTaskSensor waits for a specific task in a different DAG to finish before continuing.

Which parameter tells ExternalTaskSensor which DAG to monitor?

A execution_date

B external_task_id

C external_dag_id

D task_id

external_dag_id specifies the DAG where the sensor looks for the task.

By default, ExternalTaskSensor matches the external task's execution date with:

A The current DAG run's execution date

B The previous DAG run date

C The current system date

D The next scheduled DAG run date

It waits for the external task with the same execution date as the current DAG run.

If the external task never completes, what will ExternalTaskSensor do?

A Fail immediately

B Keep waiting until timeout or success

C Skip the task

D Trigger the external task

The sensor waits until the task succeeds or the sensor times out or fails.

Can ExternalTaskSensor wait for multiple tasks in different DAGs simultaneously?

A No, it waits for one task in one DAG only

B Yes, it supports multiple tasks natively

C Yes, but only if tasks are in the same DAG

D No, it only waits for tasks in the current DAG

It waits for one or more specific tasks in one external DAG. Multiple waits across DAGs require multiple sensors.

Explain how ExternalTaskSensor helps manage dependencies between different DAGs in Airflow.

Think about how one DAG can pause until another DAG's task finishes.

Describe what happens if the external task monitored by ExternalTaskSensor fails or never finishes.

Consider sensor behavior on task failure or delay.