0
0
Apache Airflowdevops~10 mins

Timetables for complex schedules in Apache Airflow - Step-by-Step Execution

Choose your learning style9 modes available
LearnWhyDeepVisualTryChallengeProjectRecallTime
Process Flow - Timetables for complex schedules
Define Timetable Class
↓
Create Timetable Instance
↓
Airflow Scheduler Checks Timetable
↓
Evaluate Current Time Against Timetable
↓
Trigger DAG
↓
Repeat on Schedule
Airflow uses timetable classes to decide when to run DAGs. The scheduler checks the timetable, evaluates if current time matches, then triggers the DAG or waits.
Execution Sample
Apache Airflow
from airflow.timetables.base import Timetable

class MyTimetable(Timetable):
    def next_dagrun_info(self, last_automated, now):
        # logic to define next run
        pass
Defines a custom timetable class that tells Airflow when to schedule the next DAG run.
Process Table
StepCurrent TimeLast Run TimeTimetable CheckNext Run TimeAction
12024-06-01 08:002024-06-01 00:00Is 08:00 a scheduled time?2024-06-01 08:00Trigger DAG
22024-06-01 08:052024-06-01 08:00Is 08:05 a scheduled time?2024-06-01 12:00Wait
32024-06-01 12:002024-06-01 08:00Is 12:00 a scheduled time?2024-06-01 12:00Trigger DAG
42024-06-01 12:052024-06-01 12:00Is 12:05 a scheduled time?2024-06-01 16:00Wait
52024-06-01 16:002024-06-01 12:00Is 16:00 a scheduled time?2024-06-01 16:00Trigger DAG
62024-06-01 16:052024-06-01 16:00Is 16:05 a scheduled time?2024-06-02 00:00Wait
72024-06-02 00:002024-06-01 16:00Is 00:00 a scheduled time?2024-06-02 00:00Trigger DAG
82024-06-02 00:052024-06-02 00:00Is 00:05 a scheduled time?2024-06-02 08:00Wait
92024-06-02 08:002024-06-02 00:00Is 08:00 a scheduled time?2024-06-02 08:00Trigger DAG
102024-06-02 08:052024-06-02 08:00Is 08:05 a scheduled time?2024-06-02 12:00Wait
112024-06-02 12:002024-06-02 08:00Is 12:00 a scheduled time?2024-06-02 12:00Trigger DAG
Exit2024-06-02 12:052024-06-02 12:00No more scheduled times now-Stop checking until next cycle
💡 No scheduled time matches current time; scheduler waits for next check.
Status Tracker
VariableStartAfter 1After 2After 3After 4After 5After 6After 7After 8After 9After 10After 11Final
Current Time2024-06-01 08:002024-06-01 08:052024-06-01 12:002024-06-01 12:052024-06-01 16:002024-06-01 16:052024-06-02 00:002024-06-02 00:052024-06-02 08:002024-06-02 08:052024-06-02 12:002024-06-02 12:052024-06-02 12:05
Last Run Time2024-06-01 00:002024-06-01 08:002024-06-01 08:002024-06-01 12:002024-06-01 12:002024-06-01 16:002024-06-01 16:002024-06-02 00:002024-06-02 00:002024-06-02 08:002024-06-02 08:002024-06-02 12:002024-06-02 12:00
Next Run Time2024-06-01 08:002024-06-01 12:002024-06-01 12:002024-06-01 16:002024-06-01 16:002024-06-02 00:002024-06-02 00:002024-06-02 08:002024-06-02 08:002024-06-02 12:002024-06-02 12:00--
ActionTrigger DAGWaitTrigger DAGWaitTrigger DAGWaitTrigger DAGWaitTrigger DAGWaitTrigger DAGStopStop
Key Moments - 3 Insights
Why does the scheduler wait at times like 08:05 instead of triggering the DAG?
Because the timetable only triggers DAG runs at exact scheduled times (like 08:00). At 08:05, the condition 'Is current time a scheduled time?' is false, so the scheduler waits for the next scheduled time (see execution_table rows 2 and 4).
How does the timetable know the next run time after a DAG triggers?
After triggering, the timetable calculates the next scheduled time based on its logic (e.g., every 4 hours). This is shown in the 'Next Run Time' column updating after each trigger in the execution_table.
What happens if the current time is not exactly on a scheduled time?
The scheduler does not trigger the DAG and waits. This prevents running DAGs at wrong times and ensures runs happen only at planned schedule points (see execution_table rows with 'Wait' action).
Visual Quiz - 3 Questions
Test your understanding
Look at the execution table at Step 3. What is the action taken?
AStop
BWait
CTrigger DAG
DReschedule
💡 Hint
Check the 'Action' column at Step 3 in the execution_table.
At which step does the scheduler first wait instead of triggering the DAG?
AStep 1
BStep 2
CStep 5
DStep 7
💡 Hint
Look for the first 'Wait' action in the execution_table.
If the timetable was changed to trigger every 2 hours instead of 4, how would the 'Next Run Time' change after Step 1?
ANext Run Time would be 10:00
BNext Run Time would be 12:00
CNext Run Time would be 14:00
DNext Run Time would be 08:00
💡 Hint
Consider the current schedule triggers every 4 hours; halving it means next run is 2 hours later.
Concept Snapshot
Airflow Timetables define when DAGs run.
Scheduler checks current time against timetable.
If time matches, DAG triggers; else scheduler waits.
Custom timetables allow complex schedules.
Next run time is calculated after each trigger.
Full Transcript
This visual execution shows how Airflow uses timetables to schedule DAG runs. First, a timetable class is defined with logic to decide next run times. The scheduler checks the current time against this timetable. If the current time matches a scheduled run time, the DAG triggers. Otherwise, the scheduler waits until the next scheduled time. The execution table traces times, last run, next run, and actions step-by-step. Variables like current time and last run time update as the scheduler moves through the day. Key moments clarify why the scheduler waits at non-scheduled times and how next run times are calculated. The quiz tests understanding of when DAGs trigger and how schedule changes affect next run times.