Bird
Raised Fist0
Apache Airflowdevops~7 mins

Trigger rules (all_success, one_success, none_failed) in Apache Airflow - Commands & Configuration

Choose your learning style10 modes available
Introduction
In Airflow, tasks can depend on each other. Trigger rules decide when a task should run based on the success or failure of previous tasks. This helps control the flow of work in a clear way.
When you want a task to run only if all previous tasks succeeded.
When you want a task to run if at least one previous task succeeded.
When you want a task to run as long as no previous tasks failed.
When you want to handle different outcomes of parallel tasks.
When you want to avoid running tasks if any upstream task failed.
Config File - example_dag.py
example_dag.py
from airflow import DAG
from airflow.operators.bash import BashOperator
from airflow.utils.dates import days_ago
from airflow.utils.trigger_rule import TriggerRule

default_args = {
    'start_date': days_ago(1),
}

dag = DAG(
    'trigger_rule_example',
    default_args=default_args,
    schedule_interval=None,
    catchup=False
)

# Task that always succeeds
success_task = BashOperator(
    task_id='success_task',
    bash_command='echo Success',
    dag=dag
)

# Task that always fails
fail_task = BashOperator(
    task_id='fail_task',
    bash_command='exit 1',
    dag=dag
)

# Task runs only if all upstream tasks succeed
all_success_task = BashOperator(
    task_id='all_success_task',
    bash_command='echo All upstream succeeded',
    trigger_rule=TriggerRule.ALL_SUCCESS,
    dag=dag
)

# Task runs if at least one upstream task succeeds
one_success_task = BashOperator(
    task_id='one_success_task',
    bash_command='echo At least one upstream succeeded',
    trigger_rule=TriggerRule.ONE_SUCCESS,
    dag=dag
)

# Task runs if no upstream tasks failed
none_failed_task = BashOperator(
    task_id='none_failed_task',
    bash_command='echo No upstream failed',
    trigger_rule=TriggerRule.NONE_FAILED,
    dag=dag
)

success_task >> [all_success_task, one_success_task, none_failed_task]
fail_task >> [all_success_task, one_success_task, none_failed_task]

This DAG defines five tasks:

  • success_task: always succeeds.
  • fail_task: always fails.
  • all_success_task: runs only if all upstream tasks succeed (all_success).
  • one_success_task: runs if at least one upstream task succeeds (one_success).
  • none_failed_task: runs if no upstream tasks failed (none_failed).

The last three tasks depend on both success_task and fail_task to show how trigger rules affect execution.

Commands
List all available DAGs to confirm the DAG is recognized by Airflow.
Terminal
airflow dags list
Expected OutputExpected
trigger_rule_example
Trigger the DAG run manually to start the tasks and observe trigger rules in action.
Terminal
airflow dags trigger trigger_rule_example
Expected OutputExpected
Created <DagRun trigger_rule_example @ 2024-06-01T00:00:00+00:00: manual__2024-06-01T00:00:00+00:00, externally triggered: True>
List all tasks in the DAG to verify task names and dependencies.
Terminal
airflow tasks list trigger_rule_example
Expected OutputExpected
success_task fail_task all_success_task one_success_task none_failed_task
Check the state of the success_task after DAG run to confirm it succeeded.
Terminal
airflow tasks state trigger_rule_example success_task manual__2024-06-01T00:00:00+00:00
Expected OutputExpected
success
Check the state of the fail_task after DAG run to confirm it failed.
Terminal
airflow tasks state trigger_rule_example fail_task manual__2024-06-01T00:00:00+00:00
Expected OutputExpected
failed
Check the state of all_success_task to see if it ran (it should not because one upstream failed).
Terminal
airflow tasks state trigger_rule_example all_success_task manual__2024-06-01T00:00:00+00:00
Expected OutputExpected
skipped
Check the state of one_success_task to see if it ran (it should run because one upstream succeeded).
Terminal
airflow tasks state trigger_rule_example one_success_task manual__2024-06-01T00:00:00+00:00
Expected OutputExpected
success
Check the state of none_failed_task to see if it ran (it should not because one upstream failed).
Terminal
airflow tasks state trigger_rule_example none_failed_task manual__2024-06-01T00:00:00+00:00
Expected OutputExpected
skipped
Key Concept

Trigger rules control when a task runs based on the success or failure of its upstream tasks.

Common Mistakes
Setting trigger_rule to ALL_SUCCESS but expecting the task to run when any upstream succeeds.
ALL_SUCCESS requires every upstream task to succeed, so the task will be skipped if any fail.
Use ONE_SUCCESS if you want the task to run when at least one upstream task succeeds.
Not setting trigger_rule explicitly and assuming default behavior covers all cases.
The default trigger_rule is ALL_SUCCESS, which may skip tasks if any upstream fails.
Set trigger_rule explicitly to NONE_FAILED or ONE_SUCCESS when needed.
Confusing NONE_FAILED with ALL_SUCCESS and expecting the same behavior.
NONE_FAILED runs if no upstream failed, even if some upstream tasks were skipped, unlike ALL_SUCCESS.
Choose NONE_FAILED when you want to run tasks despite skipped upstream tasks but no failures.
Summary
Define trigger rules in tasks to control execution based on upstream task results.
ALL_SUCCESS runs a task only if all upstream tasks succeed.
ONE_SUCCESS runs a task if at least one upstream task succeeds.
NONE_FAILED runs a task if no upstream tasks failed, even if some were skipped.

Practice

(1/5)
1. In Airflow, which trigger rule ensures a task runs only if all upstream tasks have succeeded?
easy
A. all_success
B. one_success
C. none_failed
D. all_failed

Solution

  1. Step 1: Understand the meaning of all_success

    This trigger rule requires every upstream task to complete successfully before the downstream task runs.
  2. Step 2: Compare with other trigger rules

    one_success runs if any upstream succeeds, none_failed runs if no upstream failed (some may be skipped), and all_failed runs only if all upstream failed.
  3. Final Answer:

    all_success -> Option A
  4. Quick Check:

    All upstream success = all_success [OK]
Hint: All upstream must succeed means all_success [OK]
Common Mistakes:
  • Confusing one_success with all_success
  • Thinking none_failed means all succeeded
  • Assuming all_failed runs on success
2. Which of the following is the correct way to set a task's trigger rule to run if at least one upstream task succeeds in Airflow DAG code?
easy
A. trigger_rule='all_success'
B. trigger_rule='all_done'
C. trigger_rule='none_failed'
D. trigger_rule='one_success'

Solution

  1. Step 1: Recall the trigger rule for running on any upstream success

    The trigger rule one_success runs the task if at least one upstream task succeeded.
  2. Step 2: Check syntax correctness

    Setting trigger_rule='one_success' is the correct syntax in Airflow task definition.
  3. Final Answer:

    trigger_rule='one_success' -> Option D
  4. Quick Check:

    At least one success = trigger_rule='one_success' [OK]
Hint: Use trigger_rule='one_success' for any upstream success [OK]
Common Mistakes:
  • Using all_success instead of one_success
  • Misspelling trigger_rule or using wrong quotes
  • Confusing none_failed with one_success
3. Given a DAG with three upstream tasks: task1 (success), task2 (skipped), task3 (failed). If a downstream task uses trigger_rule='none_failed', will it run?
medium
A. Yes, because no upstream task failed
B. No, because one upstream task failed
C. Yes, because at least one upstream succeeded
D. No, because all upstream tasks must succeed

Solution

  1. Step 1: Understand none_failed trigger rule

    This rule runs the task only if no upstream task has failed. If any upstream failed, it will not run.
  2. Step 2: Analyze upstream task states

    task3 failed, so the condition of no failures is false. Therefore, the downstream task will not run.
  3. Final Answer:

    No, because one upstream task failed -> Option B
  4. Quick Check:

    Any failure blocks none_failed run [OK]
Hint: If any upstream failed, none_failed won't run [OK]
Common Mistakes:
  • Ignoring failed upstream tasks when using none_failed
  • Confusing skipped with failed states
  • Assuming none_failed runs if some succeed
4. You have a task with trigger_rule='one_success' but it never runs even though some upstream tasks succeed. What is the likely cause?
medium
A. The task has no upstream dependencies defined
B. The DAG is paused
C. The trigger_rule is misspelled or not set correctly
D. All upstream tasks failed

Solution

  1. Step 1: Check trigger_rule spelling and assignment

    If the trigger_rule is misspelled or not properly set, Airflow defaults to all_success, causing the task not to run if not all upstream succeed.
  2. Step 2: Verify other conditions

    Since some upstream succeed, and the task has dependencies, the most common cause is incorrect trigger_rule syntax.
  3. Final Answer:

    The trigger_rule is misspelled or not set correctly -> Option C
  4. Quick Check:

    Incorrect trigger_rule causes unexpected task skipping [OK]
Hint: Check trigger_rule spelling if task doesn't run as expected [OK]
Common Mistakes:
  • Assuming upstream failure when some succeeded
  • Ignoring DAG paused state
  • Not defining upstream dependencies
5. You want a downstream task to run if either all upstream tasks succeed or if none failed but some were skipped. Which trigger rule should you use?
hard
A. none_failed
B. one_success
C. all_success
D. all_done

Solution

  1. Step 1: Understand the requirement

    The task should run if all upstream succeeded or if none failed but some were skipped. This means failures block the run, but skips do not.
  2. Step 2: Match with trigger rules

    all_success requires all succeed (no skips allowed). none_failed allows skips but no failures. one_success runs if any succeed, which is too loose. all_done runs regardless of success or failure.
  3. Step 3: Choose the correct trigger rule

    none_failed fits the requirement exactly.
  4. Final Answer:

    none_failed -> Option A
  5. Quick Check:

    No failures but allows skips = none_failed [OK]
Hint: Use none_failed to allow skips but no failures [OK]
Common Mistakes:
  • Choosing all_success which disallows skips
  • Using one_success which is too permissive
  • Confusing all_done with failure conditions
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard