Execution date vs logical date in Apache Airflow - When to Use Which

The Big Idea

Ever wondered why your daily reports sometimes show the wrong date? The secret lies in understanding execution vs logical dates!

The Scenario

Imagine you run a daily report manually every morning at 9 AM for the previous day's sales data.

You write down the date you run the report and the date the data is actually about, but sometimes you get confused which date to use for naming files or logs.

The Problem

Manually tracking when you run a task versus what data it covers is confusing and error-prone.

You might overwrite files, mix up reports, or analyze the wrong data because you used the run time instead of the data date.

The Solution

Airflow uses the execution date as the logical date (the data's date the task processes), separate from when the task actually runs.

This clear distinction helps you organize, track, and automate workflows reliably without mixing up timing and data coverage.

Before vs After

✗ Before

Run report at 9 AM and name file report_2024-06-10.csv (but unsure if date is run date or data date)

✓ After

Airflow task runs at 9 AM on 2024-06-11 with execution_date=2024-06-10 (logical date) for previous day's data

What It Enables

You can automate workflows that process data for specific dates reliably, even if tasks run later or get delayed.

Real Life Example

A daily sales ETL job runs every morning but processes yesterday's sales data using the logical date, ensuring reports always reflect the correct day.

Key Takeaways

Manual date tracking causes confusion between run time and data date.

Execution date (logical date) is the data's date; run time is when the task runs.

Airflow's separation prevents errors and improves automation reliability.