Cron expressions in Airflow - Step-by-Step Execution

Process Flow - Cron expressions in Airflow

Start

↓

Define cron expression

↓

Airflow scheduler parses expression

↓

Calculate next run time

↓

Trigger DAG run at scheduled time

↓

Wait for next schedule

↩Back to Calculate next run time

Airflow reads the cron expression, calculates when to run the task next, triggers the run, then repeats this cycle.

Execution Sample

Apache Airflow

schedule_interval = '0 6 * * *'
# Runs daily at 6:00 AM

This cron expression schedules a DAG to run every day at 6:00 in the morning.

Process Table

Step	Cron Expression Part	Value	Meaning	Next Run Time Calculation
1	Minute	0	At minute 0	Next run at HH:00
2	Hour	6	At hour 6 (6 AM)	Next run at 6:00 AM next day
3	Day of Month	*	Every day of the month	No restriction
4	Month	*	Every month	No restriction
5	Day of Week	*	Every day of the week	No restriction
6	Calculate next run	-	-	Next run scheduled at next 6:00 AM
7	Trigger DAG run	-	-	DAG runs at scheduled time
8	Wait for next schedule	-	-	Cycle repeats daily at 6:00 AM

💡 The scheduler waits until the next 6:00 AM to trigger the DAG again, repeating the cycle.

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 6	Final
Minute	-	0	0	0	0
Hour	-	-	6	6	6
Day of Month	-	-	*	*	*
Month	-	-	*	*	*
Day of Week	-	-	*	*	*
Next Run Time	None	None	None	Next 6:00 AM	Next 6:00 AM

Key Moments - 3 Insights

Why does the DAG run only once per day even though some parts of the cron expression use '*'?

The '*' means 'every' for that field, but the specific hour and minute (6:00 AM) restrict the run to once daily. See execution_table rows 2 and 3.

What happens if the cron expression is invalid?

Airflow scheduler will fail to parse it and will not schedule the DAG. This is why correct syntax is important before deployment.

How does Airflow know when to trigger the DAG next?

Airflow calculates the next run time based on the cron parts and current time, then triggers the DAG at that time. See execution_table row 6.

Visual Quiz - 3 Questions

Look at the execution table, what is the 'Hour' value at Step 2?

A *

B 0

C 6

D 12

💡 Hint

Check the 'Hour' column value in execution_table row 2.

At which step does Airflow calculate the next run time?

A Step 3

B Step 6

C Step 1

D Step 8

💡 Hint

Look for the step mentioning 'Calculate next run' in the execution_table.

If the minute part changes from '0' to '30', how does the next run time change?

A Runs daily at 6:30 AM

B Runs every minute

C Runs every hour at minute 0

D Runs only once a week

💡 Hint

Minute value controls the exact minute of the hour when the DAG runs, see variable_tracker for 'Minute'.

Concept Snapshot

Cron expressions in Airflow:
- Format: 'minute hour day_of_month month day_of_week'
- Use '*' for 'every' in a field
- Airflow scheduler parses expression to calculate next run
- DAG triggers at calculated times
- Example: '0 6 * * *' runs daily at 6:00 AM

Full Transcript

In Airflow, cron expressions define when a DAG runs. The scheduler reads the expression, breaks it into parts like minute and hour, then calculates the next time to run the DAG. For example, '0 6 * * *' means the DAG runs every day at 6:00 AM. The scheduler waits until that time, triggers the DAG, then repeats the process. Each part of the cron expression controls a time unit, and '*' means every possible value for that unit. Understanding how these parts work together helps schedule tasks precisely.