Debugging with Airflow CLI - Step-by-Step Execution

Process Flow - Debugging with Airflow CLI

Start Airflow CLI

↓

Run 'airflow tasks test'

↓

Task runs locally

↓

Check logs output

↓

Identify error or success

↓

Fix code or config

↓

Repeat test or run DAG

This flow shows how to use the Airflow CLI to run a task locally for debugging, check logs, fix issues, and repeat until the task works.

Execution Sample

Apache Airflow

airflow tasks test example_dag example_task 2024-06-01
# Runs the task locally and shows logs for debugging

This command runs a specific task from a DAG on a given date locally, showing logs to help find errors.

Process Table

Step	Command	Action	Output/Result
1	airflow tasks test example_dag example_task 2024-06-01	Run task locally	Task starts, logs show execution steps
2	Logs output	Check logs for errors or success	Logs show task running, possible error message if failure
3	Fix code/config	Edit DAG or task code based on logs	Code updated to fix error
4	Repeat test command	Run task test again	Task runs successfully, logs confirm no errors
5	Run DAG normally	Trigger DAG run in Airflow UI or CLI	DAG runs with fixed task
6	-	Exit debugging	Task works as expected, debugging complete

💡 Task runs successfully without errors, debugging finished

Status Tracker

Variable	Start	After 1	After 2	After 3	Final
Task Status	Not run	Running	Failed (error found)	Running (after fix)	Success
Logs	Empty	Execution logs appear	Error message visible	Updated logs	Clean logs, no errors

Key Moments - 3 Insights

Why do we run 'airflow tasks test' instead of triggering the whole DAG?

Running 'airflow tasks test' runs only one task locally without scheduling, so you can quickly see logs and debug without waiting for the full DAG run. See execution_table step 1.

What does it mean if logs show an error after running the test command?

It means the task code or configuration has a problem. The logs help identify the exact error so you can fix it before running the full DAG. See execution_table step 2.

Why do we repeat the test command after fixing code?

Repeating the test confirms the fix works and no new errors appear before running the DAG normally. See execution_table step 4.

Visual Quiz - 3 Questions

Look at the execution table, what is the task status after the first test run?

A Failed (error found)

B Running

C Success

D Not run

💡 Hint

Check the 'Task Status' variable in variable_tracker after step 1

At which step does the user fix the code based on logs?

A Step 2

B Step 4

C Step 3

D Step 5

💡 Hint

Look at the 'Action' column in execution_table for the step where code is edited

If the logs still show errors after step 4, what should happen next?

A Fix code again and repeat test

B Run the DAG normally

C Exit debugging

D Ignore errors and continue

💡 Hint

Refer to key_moments about repeating tests after fixes

Concept Snapshot

Use 'airflow tasks test <dag_id> <task_id> <date>' to run a task locally.
Check logs output to find errors.
Fix code or config based on logs.
Repeat test until task succeeds.
Then run DAG normally.
This speeds up debugging without full DAG runs.

Full Transcript

Debugging with Airflow CLI involves running a single task locally using the command 'airflow tasks test' with the DAG id, task id, and execution date. This runs the task immediately and shows logs in the terminal. By reading these logs, you can find errors or confirm success. If errors appear, you fix the code or configuration, then run the test command again to verify the fix. Once the task runs successfully without errors, you can trigger the full DAG run. This method helps debug tasks quickly without waiting for scheduled DAG runs.