Execution date vs logical date in Apache Airflow - CLI Comparison

Introduction

In Airflow, tasks run on schedules, and each run has two important dates: the execution date and the logical date. These dates help track when a task is supposed to run versus when it actually runs, which can be confusing at first.

When you want to understand why a task run is labeled with a date that seems earlier than the current time.

When you need to debug why data processed in a task corresponds to a previous day or period.

When scheduling daily reports that should process data from the previous day.

When setting up dependencies between tasks that rely on specific data intervals.

When monitoring backfills or reruns of past scheduled tasks.

Commands

This command triggers the DAG named 'example_dag' with an explicit execution date of June 1, 2024. The execution date represents the logical date Airflow uses to identify the run.

Terminal

airflow dags trigger example_dag --exec-date 2024-06-01T00:00:00+00:00

Expected Output

Created <DagRun example_dag @ 2024-06-01 00:00:00+00:00: manual__2024-06-01T00:00:00+00:00, externally triggered: True>

→

--exec-date - Sets the execution (logical) date for the DAG run

Lists all tasks in the 'example_dag' so you can see what tasks will run for the given execution date.

Terminal

airflow tasks list example_dag

Expected Output

task_1

task_2

task_3

Runs the task 'task_1' from 'example_dag' for the execution date June 1, 2024, simulating the run without scheduling it. This helps understand what data the task processes for that logical date.

Terminal

airflow tasks test example_dag task_1 2024-06-01

Expected Output

[2024-06-01 00:00:00,000] {taskinstance.py:xxxx} INFO - Starting attempt 1 of 1

[2024-06-01 00:00:01,000] {taskinstance.py:xxxx} INFO - Task succeeded

Shows all runs of 'example_dag' with their execution dates and states, helping you see how execution dates relate to actual run times.

Terminal

airflow dags list-runs -d example_dag

Expected Output

dag_id      execution_date          start_date             state

example_dag  2024-06-01 00:00:00    2024-06-01 00:05:00    success

→

-d - Specifies the DAG to list runs for

Key Concept

The execution date (logical date) in Airflow represents the data interval the task run is processing, not the actual time the task runs.

Common Mistakes

Confusing execution date with the current date or actual run time.

This leads to misunderstanding task logs and data processed, causing confusion about when data was handled.

Always remember execution date is the logical date for the data interval, which can be earlier than the actual run time.

Triggering DAG runs without specifying execution date and expecting it to run for the current time.

Airflow uses the execution date to determine the data interval, so omitting it can cause unexpected behavior.

Specify the execution date explicitly when triggering manual runs to control which data interval is processed.

Summary

Execution date is the logical date Airflow uses to identify the data interval for a task run.

The actual run time can be later than the execution date, which can cause confusion.

Use commands like 'airflow dags trigger' with --exec-date to control the logical date of runs.