How to fix connection not found airflow

AirflowDebug / FixBeginner · 4 min read

How to Fix 'Connection Not Found' Error in Airflow

The connection not found error in Airflow happens when a task tries to use a connection ID that Airflow cannot find in its metadata database. To fix this, ensure the connection is properly created in Airflow's UI or CLI and that the connection ID matches exactly in your DAG code.

🔍

Why This Happens

This error occurs because Airflow looks for a connection ID in its metadata database but cannot find it. This usually happens if the connection was never created, was deleted, or the connection ID used in the DAG code is misspelled.

python

from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.hooks.base import BaseHook
from datetime import datetime

def test_connection():
    conn = BaseHook.get_connection('my_conn_id')
    print(f"Connection host: {conn.host}")

default_args = {'start_date': datetime(2024, 1, 1)}

dag = DAG('example_dag', default_args=default_args, schedule_interval='@daily')

t1 = PythonOperator(
    task_id='test_conn',
    python_callable=test_connection,
    dag=dag
)

Output

airflow.exceptions.AirflowNotFoundException: The conn_id 'my_conn_id' isn't defined

🔧

The Fix

Create the missing connection in Airflow either via the web UI or CLI. Make sure the conn_id matches exactly what your DAG uses. This allows Airflow to find and use the connection without error.

bash

airflow connections add 'my_conn_id' \
    --conn-type mysql \
    --conn-host localhost \
    --conn-login user \
    --conn-password password \
    --conn-schema mydatabase

Output

Added connection 'my_conn_id' of type 'mysql'

🛡️

Prevention

Always verify connection IDs before running DAGs. Use Airflow's CLI or UI to list connections and confirm they exist. Use consistent naming conventions and consider storing connection info securely using environment variables or secrets backends.

⚠️

Related Errors

AirflowNotFoundException: Raised when a connection or resource is missing.
Authentication errors: Occur if connection credentials are wrong.
Connection timeout: Happens if Airflow cannot reach the external service.

✅

Key Takeaways

Ensure the connection ID used in your DAG matches exactly with one defined in Airflow.

Create missing connections using Airflow UI or CLI before running your DAGs.

Use consistent naming and secure storage for connection credentials.

Verify connections regularly to prevent runtime errors.

Understand related errors to troubleshoot connection issues effectively.