Authentication backends (LDAP, OAuth) in Apache Airflow - Commands & Configuration

Introduction

When you want to control who can access your Airflow web interface, you use authentication backends. LDAP and OAuth let you connect Airflow to existing login systems so users can sign in with their usual credentials.

When your company uses LDAP for employee login and you want Airflow to use the same login system.

When you want users to log in to Airflow using their Google or GitHub accounts via OAuth.

When you want to avoid managing separate usernames and passwords for Airflow.

When you want to improve security by using centralized authentication.

When you want to allow single sign-on (SSO) for Airflow users.

Config File - airflow.cfg

airflow.cfg

[webserver]
# Enable authentication
authenticate = True

# Choose the authentication backend
# For LDAP use airflow.contrib.auth.backends.ldap_auth
# For OAuth use airflow.www.fab_security.manager.OAuthUserMixin

# Example LDAP backend configuration
[ldap]
uri = ldap://ldap.example.com
user_filter = (objectClass=person)
bind_user = cn=admin,dc=example,dc=com
bind_password = admin_password
basedn = dc=example,dc=com

# Example OAuth backend configuration
[oauth]
client_id = your_client_id
client_secret = your_client_secret
authorize_url = https://accounts.google.com/o/oauth2/auth
access_token_url = https://accounts.google.com/o/oauth2/token
redirect_uri = http://localhost:8080/oauth2callback

The [webserver] section enables authentication and sets the backend type.

The [ldap] section configures connection details to your LDAP server.

The [oauth] section holds OAuth client credentials and URLs for authorization and token exchange.

You enable one backend at a time by setting the backend path in the webserver section.

Commands

Starts the Airflow webserver on port 8080 with authentication enabled using the configured backend.

Terminal

airflow webserver --port 8080

Expected Output

Serving Flask app 'airflow.www.app' (lazy loading)

 * Environment: production

 * Debug mode: off

 * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)

→

--port - Specifies the port number for the webserver

Checks the webserver response to verify that authentication is required before accessing Airflow UI.

Terminal

curl -i http://localhost:8080

Expected Output

HTTP/1.0 302 FOUND

Location: /login

Content-Type: text/html; charset=utf-8

Creates a new Airflow user named Alice with Admin role for login when using authentication backends that require user creation.

Terminal

airflow users create --username alice --firstname Alice --lastname Smith --role Admin --email [e-mail]

Expected Output

User alice created successfully

→

--role - Assigns the user role for permissions

Key Concept

If you remember nothing else from this pattern, remember: configuring Airflow to use LDAP or OAuth lets users log in with existing credentials, improving security and convenience.

Common Mistakes

Not enabling authentication in airflow.cfg (authenticate = False)

Airflow will not require login and will allow open access.

Set authenticate = True in the [webserver] section to enable login.

Using wrong backend path or missing backend configuration

Airflow will fail to authenticate users or fallback to no authentication.

Set the correct backend path in airflow.cfg and provide all required settings for LDAP or OAuth.

Not creating users when required by the backend

Users cannot log in if they do not exist in Airflow's user database.

Use airflow users create command to add users before login.

Summary

Enable authentication in airflow.cfg by setting authenticate = True.

Configure either LDAP or OAuth backend with correct connection and credential details.

Start the Airflow webserver and verify login is required.

Create users if the backend requires it for login.

Practice

1. What is the main purpose of configuring an authentication backend like LDAP or OAuth in Airflow?

easy

A. To schedule DAGs automatically

B. To verify users using external systems for better security

C. To store Airflow logs externally

D. To speed up Airflow task execution

Solution

1. Step 1: Understand authentication backend role

   Authentication backends connect Airflow to external user verification systems like LDAP or OAuth.

2. Step 2: Identify the main benefit

   This setup improves security and user management by using existing accounts outside Airflow.

3. Final Answer:

   To verify users using external systems for better security -> Option B

4. Quick Check:

   Authentication backend = user verification [OK]

Hint: Authentication backends verify users externally [OK]

Common Mistakes:

• Confusing authentication with task scheduling
• Thinking it speeds up task execution
• Assuming it stores logs

2. Which of the following is the correct way to specify the LDAP authentication backend in the airflow.cfg file?

easy

A. auth_backend = airflow.providers.ldap.auth.backend.LDAPAuthBackend

B. auth_backend = airflow.contrib.auth.backends.ldap_auth

C. auth_backend = airflow.www.auth.backend.ldap

D. auth_backend = airflow.contrib.auth.backends.ldap_auth_backend.LDAPBackend

Solution

1. Step 1: Recall the correct LDAP backend path

   The correct LDAP backend path in Airflow 2.x is airflow.providers.ldap.auth.backend.LDAPAuthBackend.

2. Step 2: Compare options

   auth_backend = airflow.providers.ldap.auth.backend.LDAPAuthBackend matches the correct syntax exactly; others have incorrect module paths or class names.

3. Final Answer:

   auth_backend = airflow.providers.ldap.auth.backend.LDAPAuthBackend -> Option A

4. Quick Check:

   Correct LDAP backend path = auth_backend = airflow.providers.ldap.auth.backend.LDAPAuthBackend [OK]

Hint: Use airflow.providers.ldap.auth.backend.LDAPAuthBackend for LDAP [OK]

Common Mistakes:

• Using incorrect module paths
• Adding extra class names
• Confusing OAuth backend with LDAP

3. Given this snippet in airflow.cfg:

[webserver]
auth_backend = airflow.contrib.auth.backends.oauth_auth

What will happen when a user tries to log in?

medium

A. Airflow will allow login without authentication

B. Airflow will reject all logins because OAuth is not supported

C. Airflow will use LDAP authentication instead

D. Airflow will authenticate the user using OAuth provider settings

Solution

1. Step 1: Identify the configured backend

   The config sets auth_backend to airflow.contrib.auth.backends.oauth_auth, which enables OAuth authentication.

2. Step 2: Understand login behavior

   With OAuth backend, Airflow redirects login to the OAuth provider for user verification.

3. Final Answer:

   Airflow will authenticate the user using OAuth provider settings -> Option D

4. Quick Check:

   OAuth backend means OAuth login [OK]

Hint: OAuth backend means login via OAuth provider [OK]

Common Mistakes:

• Assuming OAuth is unsupported
• Confusing OAuth with LDAP
• Thinking login is disabled

4. You configured LDAP authentication in airflow.cfg but users cannot log in. Which of these is the most likely cause?

medium

A. The Airflow scheduler is not running

B. The DAGs folder is empty

C. The auth_backend setting points to a non-existent module

D. The webserver port is set to 8080

Solution

1. Step 1: Analyze login failure causes

   If users cannot log in after LDAP setup, a common cause is a wrong or missing auth_backend module path.

2. Step 2: Eliminate unrelated causes

   Empty DAGs, scheduler status, or port settings do not affect authentication backend functionality.

3. Final Answer:

   The auth_backend setting points to a non-existent module -> Option C

4. Quick Check:

   Wrong auth_backend module = login failure [OK]

Hint: Check auth_backend module path if login fails [OK]

Common Mistakes:

• Blaming DAGs folder for login issues
• Confusing scheduler status with auth problems
• Thinking port affects authentication

5. You want Airflow to authenticate users via OAuth and also allow admin users to log in with LDAP. Which approach is best to achieve this?

hard

A. Configure a custom authentication backend that combines OAuth and LDAP logic

B. Set auth_backend to LDAP only and ignore OAuth

C. Use OAuth backend and manually add admin users in Airflow database

D. Run two Airflow instances, one with LDAP and one with OAuth

Solution

1. Step 1: Understand multi-backend authentication needs

   Airflow supports one auth_backend at a time, so combining OAuth and LDAP requires custom code.

2. Step 2: Evaluate options

   Configure a custom authentication backend that combines OAuth and LDAP logic suggests creating a custom backend to handle both OAuth and LDAP, which is the correct approach.

3. Step 3: Reject other options

   Setting auth_backend to LDAP only, using OAuth backend and manually adding admin users, or running two Airflow instances do not support both methods simultaneously; the latter is also complex and inefficient.

4. Final Answer:

   Configure a custom authentication backend that combines OAuth and LDAP logic -> Option A

5. Quick Check:

   Multi-auth needs custom backend [OK]

Hint: Combine OAuth and LDAP with custom backend code [OK]

Common Mistakes:

• Trying to set two auth_backends in airflow.cfg
• Ignoring admin user needs
• Running multiple Airflow instances unnecessarily