ExternalTaskSensor for cross-DAG dependencies in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

What is the output when ExternalTaskSensor waits for a non-existent DAG?

You have an ExternalTaskSensor configured to wait for a task in a DAG that does not exist. What will happen when the sensor runs?

Apache Airflow

sensor = ExternalTaskSensor(
    task_id='wait_for_task',
    external_dag_id='non_existent_dag',
    external_task_id='some_task',
    mode='reschedule',
    timeout=30
)

sensor.execute(context={})

A The sensor completes successfully without waiting because the external DAG is missing.

B The sensor raises AirflowException immediately because the external DAG is not found.

C The sensor waits until timeout and then raises AirflowSensorTimeout.

D The sensor retries indefinitely without timeout.

💡 Hint

Think about how sensors behave when the external task or DAG is not yet available.

ExternalTaskSensor checks for the external DAG on the first poke. If the DAG does not exist, it raises AirflowException immediately.

🧠 Conceptual

intermediate

Which parameter controls the mode of ExternalTaskSensor to free worker slots while waiting?

In Airflow's ExternalTaskSensor, which parameter should you set to 'reschedule' mode to avoid occupying a worker slot while waiting for the external task?

A mode

B soft_fail

C timeout

D poke_interval

💡 Hint

This parameter changes how the sensor behaves during waiting periods.

The 'mode' parameter controls whether the sensor occupies a worker slot ('poke') or frees it while waiting ('reschedule'). Setting mode='reschedule' frees the slot.

🔀 Workflow

advanced

Identify the correct ExternalTaskSensor configuration for cross-DAG dependency with execution date tolerance

You want to configure an ExternalTaskSensor to wait for a task in another DAG but allow a 10-minute delay in execution date matching. Which configuration is correct?

A ExternalTaskSensor(external_dag_id='dag_b', external_task_id='task_b', execution_date_fn=lambda dt: dt - timedelta(minutes=10))

B ExternalTaskSensor(external_dag_id='dag_b', external_task_id='task_b', execution_delta=timedelta(minutes=-10))

C ExternalTaskSensor(external_dag_id='dag_b', external_task_id='task_b', execution_date_fn=lambda dt: dt + timedelta(minutes=10))

D ExternalTaskSensor(external_dag_id='dag_b', external_task_id='task_b', execution_delta=timedelta(minutes=10))

💡 Hint

execution_delta shifts the execution date to look for the external task.

execution_delta shifts the execution date backward by 10 minutes, allowing the sensor to wait for a task that ran up to 10 minutes earlier.

❓ Troubleshoot

advanced

Why does ExternalTaskSensor fail with 'No active runs found' error?

You configured ExternalTaskSensor to wait for a task in another DAG, but it fails with 'No active runs found' error. What is the most likely cause?

A The external DAG has no active DAG runs matching the execution date the sensor is waiting for.

B The external task_id is misspelled in the sensor configuration.

C The sensor's poke_interval is too short causing premature failure.

D The sensor is running in 'poke' mode instead of 'reschedule' mode.

💡 Hint

Check the external DAG's run status and execution dates.

The sensor looks for an active DAG run matching the execution date. If none exists, it raises 'No active runs found'.

✅ Best Practice

expert

What is the best practice to avoid deadlocks when using ExternalTaskSensor for cross-DAG dependencies?

When using ExternalTaskSensor to create dependencies between DAGs, what is the best practice to avoid deadlocks and ensure smooth execution?

A Use mode='poke' to keep the sensor active and avoid missing external task completion.

B Set timeout to None so the sensor waits indefinitely for the external task.

C Use execution_delta or execution_date_fn to align execution dates and avoid circular waits.

D Trigger all dependent DAGs manually to avoid sensor waits.

💡 Hint

Think about how execution dates affect cross-DAG dependencies and waiting.

Aligning execution dates with execution_delta or execution_date_fn prevents sensors from waiting for tasks that never run, avoiding deadlocks.