How to Fix Scheduler Not Running in Airflow: Quick Solutions

If the Airflow scheduler is not running, first check that the scheduler process is started with airflow scheduler and running in the background. Also, verify the airflow.cfg configuration for correct database connection and that the metadata database is reachable and migrated.

Why This Happens

The Airflow scheduler may not run because the scheduler process is not started or crashes immediately. Common causes include incorrect database connection settings, missing database migrations, or scheduler process conflicts.

For example, if the database URL in airflow.cfg is wrong, the scheduler cannot connect and will fail silently or exit.

ini

[scheduler]
# Incorrect database connection string
sql_alchemy_conn = postgresql+psycopg2://user:wrongpassword@localhost:5432/airflow_db

# Scheduler command not started
# No running scheduler process

Output

Scheduler exits immediately or logs errors like 'Could not connect to database' or 'OperationalError'. No scheduled tasks run.

The Fix

Fix the database connection string in airflow.cfg to the correct credentials and ensure the metadata database is reachable. Run airflow db upgrade to apply any missing migrations. Then start the scheduler with airflow scheduler and keep it running in the background or as a service.

bash

[scheduler]
sql_alchemy_conn = postgresql+psycopg2://user:correctpassword@localhost:5432/airflow_db

# Start scheduler
$ airflow db upgrade
$ airflow scheduler &

Output

Scheduler started successfully and begins scheduling DAG runs without errors.

Prevention

To avoid scheduler issues, always verify your airflow.cfg database settings after installation or upgrades. Use airflow db check to test database connectivity. Run airflow db upgrade after Airflow upgrades to keep the metadata database schema updated. Run the scheduler as a managed service or use process managers like systemd or supervisord to keep it alive.

Related Errors

Other common errors include:

• Scheduler stuck in 'running' state: Usually caused by DAG parsing errors or heavy load. Fix by checking DAG code and logs.
• Scheduler not picking up new DAGs: Caused by incorrect DAG folder path or permissions. Fix by verifying dags_folder and file permissions.

Key Takeaways

Always verify and correct the database connection string in airflow.cfg.

Run airflow db upgrade after Airflow installation or upgrades to update the metadata database.

Start the scheduler process with airflow scheduler and keep it running continuously.

Use process managers to ensure the scheduler stays alive and restarts on failure.

Check scheduler logs for errors to diagnose connection or DAG parsing issues.