Trigger rules (all_success, one_success, none_failed) in Apache Airflow - Step-by-Step Execution

Process Flow - Trigger rules (all_success, one_success, none_failed)

Start Task Execution

↓

Check Upstream Tasks Status

↓

all_success?

Yes→Run Task

↓

Skip Task

↓

one_success?

Yes→Run Task

↓

Skip Task

↓

none_failed?

Yes→Run Task

No↓

Skip Task

The task checks upstream tasks' statuses and decides to run or skip based on the trigger rule: all_success requires all upstream tasks to succeed, one_success requires at least one success, none_failed requires no failures.

Execution Sample

Apache Airflow

task = PythonOperator(
    task_id='example',
    python_callable=my_func,
    trigger_rule='all_success'
)

This task runs only if all upstream tasks succeed.

Process Table

Step	Upstream Tasks Status	Trigger Rule	Condition Met?	Action
1	Task A: success, Task B: success	all_success	Yes	Run Task
2	Task A: success, Task B: failed	all_success	No	Skip Task
3	Task A: success, Task B: failed	one_success	Yes	Run Task
4	Task A: failed, Task B: failed	one_success	No	Skip Task
5	Task A: success, Task B: skipped	none_failed	Yes	Run Task
6	Task A: success, Task B: failed	none_failed	No	Skip Task
7	Task A: skipped, Task B: skipped	none_failed	Yes	Run Task

💡 Execution stops after deciding to run or skip the task based on the trigger rule condition.

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	After Step 7
Upstream Statuses	N/A	success, success	success, failed	success, failed	failed, failed	success, skipped	success, failed	skipped, skipped
Trigger Rule	all_success	all_success	all_success	one_success	one_success	none_failed	none_failed	none_failed
Condition Met	N/A	Yes	No	Yes	No	Yes	No	Yes
Action	N/A	Run	Skip	Run	Skip	Run	Skip	Run

Key Moments - 3 Insights

Why does the task skip when one upstream task fails under 'all_success'?

Because 'all_success' requires every upstream task to succeed. The execution_table rows 1 and 2 show that if any upstream task fails, the condition is not met, so the task is skipped.

How can a task run if one upstream task is skipped under 'none_failed'?

The 'none_failed' rule allows the task to run as long as no upstream task failed. Skipped tasks do not count as failures. See execution_table row 5 where one task is skipped but the task still runs.

What happens if all upstream tasks fail under 'one_success'?

The task will skip because 'one_success' requires at least one success. Execution_table row 4 shows that with all failures, the condition is not met and the task is skipped.

Visual Quiz - 3 Questions

Look at the execution_table, at step 3, what is the action taken?

A Run Task

B Skip Task

C Retry Task

D Fail Task

💡 Hint

Check the 'Action' column in execution_table row 3 where upstream tasks are success and failed with 'one_success' trigger.

At which step does the 'all_success' trigger rule cause the task to skip?

A Step 1

B Step 5

C Step 2

D Step 7

💡 Hint

Look at execution_table rows with 'all_success' trigger and find where 'Condition Met?' is 'No'.

If Task B status changes from 'failed' to 'success' at step 6, what would be the new action for 'none_failed'?

A Skip Task

B Run Task

C Fail Task

D Retry Task

💡 Hint

Refer to variable_tracker for step 6 and understand that 'none_failed' requires no failures to run.

Concept Snapshot

Trigger rules decide if a task runs based on upstream tasks' states.
'all_success': run if all upstream succeed.
'one_success': run if at least one upstream succeeds.
'none_failed': run if no upstream failed (skipped allowed).
If condition not met, task is skipped.

Full Transcript

Trigger rules in Airflow control when a task runs based on the status of upstream tasks. The 'all_success' rule requires every upstream task to succeed before running the task. If any upstream task fails, the task is skipped. The 'one_success' rule runs the task if at least one upstream task succeeds; if all fail, the task is skipped. The 'none_failed' rule allows the task to run as long as no upstream task has failed; skipped tasks do not count as failures. The execution table shows different upstream task status combinations and the resulting action for each trigger rule. The variable tracker records how upstream statuses, trigger rules, conditions, and actions change step by step. Key moments clarify common confusions about why tasks run or skip under different rules. The visual quiz tests understanding by asking about specific steps and outcomes. The concept snapshot summarizes the trigger rules and their behavior in a quick reference format.

Practice

1. In Airflow, which trigger rule ensures a task runs only if all upstream tasks have succeeded?

easy

A. all_success

B. one_success

C. none_failed

D. all_failed

Solution

1. Step 1: Understand the meaning of all_success

   This trigger rule requires every upstream task to complete successfully before the downstream task runs.

2. Step 2: Compare with other trigger rules

   one_success runs if any upstream succeeds, none_failed runs if no upstream failed (some may be skipped), and all_failed runs only if all upstream failed.

3. Final Answer:

   all_success -> Option A

4. Quick Check:

   All upstream success = all_success [OK]

Hint: All upstream must succeed means all_success [OK]

Common Mistakes:

• Confusing one_success with all_success
• Thinking none_failed means all succeeded
• Assuming all_failed runs on success

2. Which of the following is the correct way to set a task's trigger rule to run if at least one upstream task succeeds in Airflow DAG code?

easy

A. trigger_rule='all_success'

B. trigger_rule='all_done'

C. trigger_rule='none_failed'

D. trigger_rule='one_success'

Solution

1. Step 1: Recall the trigger rule for running on any upstream success

   The trigger rule one_success runs the task if at least one upstream task succeeded.

2. Step 2: Check syntax correctness

   Setting trigger_rule='one_success' is the correct syntax in Airflow task definition.

3. Final Answer:

   trigger_rule='one_success' -> Option D

4. Quick Check:

   At least one success = trigger_rule='one_success' [OK]

Hint: Use trigger_rule='one_success' for any upstream success [OK]

Common Mistakes:

• Using all_success instead of one_success
• Misspelling trigger_rule or using wrong quotes
• Confusing none_failed with one_success

3. Given a DAG with three upstream tasks: task1 (success), task2 (skipped), task3 (failed). If a downstream task uses trigger_rule='none_failed', will it run?

medium

A. Yes, because no upstream task failed

B. No, because one upstream task failed

C. Yes, because at least one upstream succeeded

D. No, because all upstream tasks must succeed

Solution

1. Step 1: Understand none_failed trigger rule

   This rule runs the task only if no upstream task has failed. If any upstream failed, it will not run.

2. Step 2: Analyze upstream task states

   task3 failed, so the condition of no failures is false. Therefore, the downstream task will not run.

3. Final Answer:

   No, because one upstream task failed -> Option B

4. Quick Check:

   Any failure blocks none_failed run [OK]

Hint: If any upstream failed, none_failed won't run [OK]

Common Mistakes:

• Ignoring failed upstream tasks when using none_failed
• Confusing skipped with failed states
• Assuming none_failed runs if some succeed

4. You have a task with trigger_rule='one_success' but it never runs even though some upstream tasks succeed. What is the likely cause?

medium

A. The task has no upstream dependencies defined

B. The DAG is paused

C. The trigger_rule is misspelled or not set correctly

D. All upstream tasks failed

Solution

1. Step 1: Check trigger_rule spelling and assignment

   If the trigger_rule is misspelled or not properly set, Airflow defaults to all_success, causing the task not to run if not all upstream succeed.

2. Step 2: Verify other conditions

   Since some upstream succeed, and the task has dependencies, the most common cause is incorrect trigger_rule syntax.

3. Final Answer:

   The trigger_rule is misspelled or not set correctly -> Option C

4. Quick Check:

   Incorrect trigger_rule causes unexpected task skipping [OK]

Hint: Check trigger_rule spelling if task doesn't run as expected [OK]

Common Mistakes:

• Assuming upstream failure when some succeeded
• Ignoring DAG paused state
• Not defining upstream dependencies

5. You want a downstream task to run if either all upstream tasks succeed or if none failed but some were skipped. Which trigger rule should you use?

hard

A. none_failed

B. one_success

C. all_success

D. all_done

Solution

1. Step 1: Understand the requirement

   The task should run if all upstream succeeded or if none failed but some were skipped. This means failures block the run, but skips do not.

2. Step 2: Match with trigger rules

   all_success requires all succeed (no skips allowed). none_failed allows skips but no failures. one_success runs if any succeed, which is too loose. all_done runs regardless of success or failure.

3. Step 3: Choose the correct trigger rule

   none_failed fits the requirement exactly.

4. Final Answer:

   none_failed -> Option A

5. Quick Check:

   No failures but allows skips = none_failed [OK]

Hint: Use none_failed to allow skips but no failures [OK]

Common Mistakes:

• Choosing all_success which disallows skips
• Using one_success which is too permissive
• Confusing all_done with failure conditions