Where to Put DAG Files in Airflow: Location and Best Practices

Place your DAG files inside the directory specified by the dags_folder setting in your Airflow configuration file (airflow.cfg). By default, this is the ~/airflow/dags folder, but you can customize it to any accessible path.

Syntax

The dags_folder setting in airflow.cfg defines where Airflow looks for DAG files. This path must be a directory containing your Python scripts that define DAGs.

Example path in airflow.cfg:

• dags_folder = /path/to/your/dags

Each DAG file inside this folder should be a valid Python file defining one or more DAG objects.

ini

[core]
dags_folder = /path/to/your/dags

Example

This example shows a simple DAG file placed inside the default dags folder. Airflow will automatically detect and schedule this DAG.

python

from airflow import DAG
from airflow.operators.bash import BashOperator
from datetime import datetime

default_args = {
    'start_date': datetime(2024, 1, 1),
}

dag = DAG(
    'example_dag',
    default_args=default_args,
    schedule_interval='@daily',
    catchup=False
)

task = BashOperator(
    task_id='print_date',
    bash_command='date',
    dag=dag
)

Output

No direct output; Airflow scheduler detects and runs the DAG daily, executing the 'print_date' task.

Common Pitfalls

Common mistakes when placing DAG files include:

• Putting DAG files outside the dags_folder so Airflow cannot find them.
• Using non-Python files or files without valid DAG definitions.
• Not restarting the Airflow scheduler after adding new DAG files, so changes are not detected.
• Setting dags_folder to a path without proper read permissions.

Always verify your airflow.cfg and ensure your DAG files are valid Python scripts inside the configured folder.

bash

## Wrong: DAG file outside dags_folder
# File placed in /home/user/scripts/dag.py but dags_folder is ~/airflow/dags

## Right: DAG file inside dags_folder
# File placed in ~/airflow/dags/example_dag.py

Quick Reference

• Default DAGs folder: ~/airflow/dags
• Config file: airflow.cfg under [core] section
• Change DAGs folder: Edit dags_folder path in airflow.cfg
• Restart scheduler: Required after adding or changing DAG files
• File type: Python scripts defining DAG objects

Key Takeaways

Put DAG files inside the directory set by the dags_folder in airflow.cfg.

By default, this folder is ~/airflow/dags but can be customized.

DAG files must be valid Python scripts defining DAG objects.

Restart the Airflow scheduler after adding or changing DAG files.

Ensure the dags_folder path has proper read permissions.