Secrets management in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the primary purpose of Airflow's secrets backend?

Airflow supports multiple secrets backends like HashiCorp Vault, AWS Secrets Manager, and environment variables. What is the main reason to use a secrets backend in Airflow?

A To automatically generate logs for task failures.

B To speed up DAG parsing by caching task metadata.

C To securely store and retrieve sensitive information like passwords and API keys during task execution.

D To schedule DAG runs based on secret expiration dates.

💡 Hint

Think about why you wouldn't want to hardcode passwords in your DAG files.

Secrets backends in Airflow are designed to keep sensitive data safe and provide it only when needed during task runs.

💻 Command Output

intermediate

What is the output of this Airflow CLI command?

Given that Airflow is configured with the environment variables secrets backend, what will this command output?

Apache Airflow

airflow secrets get --conn-id my_postgres_conn

A Empty output because secrets backends do not support CLI retrieval

B A JSON object with all connection details printed to the console

C SyntaxError: invalid command usage

D Connection URI string stored in the environment variable AIRFLOW_CONN_MY_POSTGRES_CONN

💡 Hint

Check how Airflow environment variable secrets are named and accessed.

The CLI command retrieves the connection URI from the environment variable named AIRFLOW_CONN_MY_POSTGRES_CONN when using the environment variables secrets backend.

❓ Configuration

advanced

Which configuration snippet correctly enables HashiCorp Vault as Airflow's secrets backend?

Choose the correct airflow.cfg snippet to configure HashiCorp Vault as the secrets backend with the Vault URL at https://vault.example.com and token authentication.

A

[secrets]
backend = airflow.providers.hashicorp.vault.VaultBackend
backend_kwargs = {"url": "vault.example.com", "auth": "token"}

B

[secrets]
backend = airflow.providers.hashicorp.secrets.vault.VaultBackend
backend_kwargs = {"url": "https://vault.example.com", "token": "mytoken"}

C

[secrets]
backend = airflow.secrets.vault.VaultBackend
backend_kwargs = {"url": "https://vault.example.com", "token": "mytoken"}

D

[secrets]
backend = airflow.providers.hashicorp.secrets.vault.VaultBackend
backend_kwargs = {"endpoint": "https://vault.example.com", "token": "mytoken"}

💡 Hint

Check the exact import path and parameter names for VaultBackend in Airflow providers.

The correct backend path is airflow.providers.hashicorp.secrets.vault.VaultBackend and the backend_kwargs keys must match the expected parameters like 'url' and 'token'.

❓ Troubleshoot

advanced

Why does Airflow fail to retrieve secrets from AWS Secrets Manager?

An Airflow deployment uses AWS Secrets Manager as the secrets backend. The DAGs fail with errors saying secrets are missing. Which of these is the most likely cause?

A The IAM role or user running Airflow lacks permissions to read secrets from AWS Secrets Manager.

B The secrets backend is not supported by Airflow and must be disabled.

C The secrets are stored in AWS S3 instead of AWS Secrets Manager.

D Airflow requires secrets to be stored only in environment variables.

💡 Hint

Think about AWS permissions needed for Airflow to access secrets.

Airflow needs proper AWS IAM permissions to read secrets from AWS Secrets Manager. Without them, it cannot retrieve secrets and will fail.

🔀 Workflow

expert

What is the correct order to securely use secrets in an Airflow DAG?

Arrange these steps in the correct order to securely use a database password stored in a secrets backend within an Airflow DAG.

A 1,2,3,4

B 2,1,3,4

C 1,3,2,4

D 3,1,2,4

💡 Hint

Think about what must exist before Airflow can retrieve secrets and how tasks use them.

First, the secret must be stored securely. Then Airflow must be configured to access it. Next, DAGs reference the secret, and finally tasks use it without hardcoding.