0
0
Apache Airflowdevops~10 mins

Execution date vs logical date in Apache Airflow - Visual Side-by-Side Comparison

Choose your learning style9 modes available
LearnWhyDeepVisualTryChallengeProjectRecallTime
Process Flow - Execution date vs logical date
DAG scheduled at time T
↓
Execution Date = T - schedule interval
↓
Logical Date = Execution Date
↓
Task runs using Logical Date
↓
Next DAG run scheduled at T + schedule interval
Shows how Airflow schedules a DAG run at time T but assigns an execution date that is the previous schedule interval, called the logical date, which tasks use for data partitioning.
Execution Sample
Apache Airflow
from datetime import datetime, timedelta

schedule_interval = '@daily'
run_time = datetime(2024, 6, 2, 0, 0)
execution_date = run_time - timedelta(days=1)
logical_date = execution_date
print(f"Run at {run_time.strftime('%Y-%m-%d %H:%M')}, execution_date: {execution_date.strftime('%Y-%m-%d %H:%M')}, logical_date: {logical_date.strftime('%Y-%m-%d %H:%M')}")
This code simulates a daily DAG run at midnight June 2, 2024, showing execution and logical dates as June 1, 2024.
Process Table
StepRun Time (Actual)Execution DateLogical DateExplanation
12024-06-01 00:002024-05-31 00:002024-05-31 00:00First DAG run scheduled at June 1 midnight, execution date is previous day midnight
22024-06-02 00:002024-06-01 00:002024-06-01 00:00Second DAG run scheduled at June 2 midnight, execution date is June 1 midnight
32024-06-03 00:002024-06-02 00:002024-06-02 00:00Third DAG run scheduled at June 3 midnight, execution date is June 2 midnight
42024-06-04 00:002024-06-03 00:002024-06-03 00:00Fourth DAG run scheduled at June 4 midnight, execution date is June 3 midnight
52024-06-05 00:002024-06-04 00:002024-06-04 00:00Fifth DAG run scheduled at June 5 midnight, execution date is June 4 midnight
6N/AN/AN/ANo more runs in this example
💡 DAG runs at midnight each day, execution date is always previous day midnight, which is the logical date tasks use.
Status Tracker
VariableStartAfter 1After 2After 3After 4After 5
run_time2024-06-01 00:002024-06-02 00:002024-06-03 00:002024-06-04 00:002024-06-05 00:00N/A
execution_date2024-05-31 00:002024-06-01 00:002024-06-02 00:002024-06-03 00:002024-06-04 00:00N/A
logical_date2024-05-31 00:002024-06-01 00:002024-06-02 00:002024-06-03 00:002024-06-04 00:00N/A
Key Moments - 3 Insights
Why is the execution date one day before the actual run time?
Because Airflow schedules DAG runs at the end of the period, the execution date represents the start of the data interval. See execution_table rows 1-5 where execution_date is always one day before run_time.
Are execution date and logical date different?
No, in Airflow they are the same. Logical date is just another name for execution date used in task context. This is shown in the execution_table where both columns match exactly.
What happens if the schedule interval changes?
The gap between run_time and execution_date changes accordingly. For example, with '@hourly' schedule, execution_date is one hour before run_time. This example uses '@daily' so the gap is one day.
Visual Quiz - 3 Questions
Test your understanding
Look at the execution_table at step 3. What is the execution date?
A2024-06-03 00:00
B2024-06-02 00:00
C2024-06-01 00:00
D2024-06-04 00:00
💡 Hint
Check the 'Execution Date' column in row 3 of the execution_table.
At which run time does the execution date equal 2024-06-01 00:00?
A2024-06-01 00:00
B2024-06-03 00:00
C2024-06-02 00:00
D2024-06-04 00:00
💡 Hint
Look for execution_date = 2024-06-01 00:00 in the execution_table and find corresponding run_time.
If the schedule interval changed to '@hourly', how would execution_date relate to run_time?
Aexecution_date would be one hour before run_time
Bexecution_date would be the same as run_time
Cexecution_date would be one day after run_time
Dexecution_date would be two hours before run_time
💡 Hint
Recall the explanation in key_moments about how execution_date is run_time minus schedule interval.
Concept Snapshot
Airflow schedules DAG runs at fixed intervals.
Execution date is the start of the data interval, usually run_time minus schedule interval.
Logical date is another name for execution date.
Tasks use logical date to process data for that period.
This helps organize data pipelines by data intervals, not run times.
Full Transcript
In Airflow, the execution date is the date assigned to a DAG run that represents the start of the data interval it processes. Although the DAG runs at a scheduled time, called run_time, the execution date is usually the run_time minus the schedule interval. This execution date is also called the logical date. For example, a daily DAG scheduled to run at midnight on June 2, 2024, has an execution date of June 1, 2024. This means the DAG run processes data for June 1. The logical date is used inside tasks to reference the data partition. This concept helps keep data pipelines organized by the data they process rather than the time they run. The execution_table shows this step by step for five daily runs. The variable_tracker shows how run_time, execution_date, and logical_date change over runs. Key moments clarify common confusions about why execution date is before run time and the equivalence of execution and logical dates. The visual quiz tests understanding of these dates in the table and how schedule interval affects them.