Connection encryption in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Airflow Connection Encryption Purpose

Why is connection encryption important in Apache Airflow?

A It protects sensitive connection credentials from being exposed in the Airflow metadata database.

B It speeds up the execution of DAGs by compressing connection data.

C It allows Airflow to connect to external APIs without authentication.

D It automatically retries failed tasks using encrypted channels.

💡 Hint

Think about what sensitive information Airflow stores and why encryption matters.

Airflow stores connection credentials like passwords and tokens in its metadata database. Encrypting these credentials prevents unauthorized users from reading them if they access the database.

❓ Configuration

intermediate

Enabling Connection Encryption in Airflow Configuration

Which configuration setting enables connection encryption in Airflow?

A Set <code>encrypt_connections = True</code> in <code>airflow.cfg</code>.

B Add <code>connection_encryption = enabled</code> in the DAG file.

C Set <code>fernet_key</code> in <code>airflow.cfg</code> under the <code>core</code> section.

D Set <code>enable_ssl = True</code> in the <code>webserver</code> section.

💡 Hint

Look for a key related to encryption in the main Airflow config file.

Airflow uses a fernet_key in airflow.cfg to encrypt and decrypt connection passwords. This key must be set under the [core] section.

💻 Command Output

advanced

Output of Airflow Connections List with Encryption Enabled

What is the output of the command airflow connections list when connection encryption is enabled and passwords are set?

Apache Airflow

airflow connections list

A The passwords are shown as encrypted strings (e.g., <code>gAAAAAB...</code>).

B Passwords are replaced with <code>***</code> and not shown.

C The command fails with a decryption error.

D The passwords are shown in plain text.

💡 Hint

Think about how Airflow stores and displays encrypted passwords.

When encryption is enabled, Airflow stores passwords encrypted but decrypts them when displaying. However, the CLI shows the encrypted string to avoid exposing plain text.

❓ Troubleshoot

advanced

Troubleshooting Connection Decryption Failures

You enabled connection encryption by setting a new fernet_key but now Airflow fails to decrypt existing connection passwords. What is the cause?

A The database connection string is incorrect.

B The Airflow version does not support encryption.

C The <code>airflow.cfg</code> file is missing the <code>executor</code> setting.

D The new <code>fernet_key</code> does not match the key used to encrypt existing passwords.

💡 Hint

Think about how encryption keys work for decrypting data.

If you change the fernet_key, Airflow cannot decrypt passwords encrypted with the old key. The key must remain consistent to decrypt existing data.

✅ Best Practice

expert

Best Practice for Managing Airflow Fernet Keys

What is the best practice for managing the fernet_key used for connection encryption in Airflow?

A Generate a new <code>fernet_key</code> every time Airflow starts to maximize security.

B Store the <code>fernet_key</code> securely outside the codebase and rotate it carefully without losing access to encrypted data.

C Hardcode the <code>fernet_key</code> in DAG files for easy access.

D Disable encryption and rely on database user permissions instead.

💡 Hint

Consider security and data availability when handling encryption keys.

The fernet_key must be stored securely and rotated carefully. Losing the key means losing access to encrypted connection passwords. Generating a new key every start or hardcoding it in DAGs is insecure or impractical.

Practice

1. What is the main purpose of enabling connection encryption in Airflow?

easy

A. To protect data exchanged between Airflow and external systems

B. To speed up the connection between Airflow and databases

C. To reduce the size of data transferred over the network

D. To allow Airflow to connect without authentication

Solution

1. Step 1: Understand connection encryption

   Encryption secures data by making it unreadable to outsiders during transfer.

2. Step 2: Apply to Airflow connections

   Airflow uses encryption to protect sensitive data when communicating with external systems.

3. Final Answer:

   To protect data exchanged between Airflow and external systems -> Option A

4. Quick Check:

   Encryption protects data [OK]

Hint: Encryption means data is safe from outsiders [OK]

Common Mistakes:

• Thinking encryption speeds up connections
• Confusing encryption with data compression
• Assuming encryption removes authentication

2. Which of the following is the correct way to specify SSL encryption in an Airflow connection's extra field?

easy

A. {"sslmode": "require"}

B. {"ssl": "false"}

C. {"encrypt": "no"}

D. {"ssl_enabled": "off"}

Solution

1. Step 1: Identify correct SSL option key

   The standard key for SSL mode in Airflow connection extras is "sslmode" with values like "require".

2. Step 2: Check option values

   "require" enables SSL, while other options like "false", "no", or "off" disable encryption.

3. Final Answer:

   {"sslmode": "require"} -> Option A

4. Quick Check:

   Use "sslmode": "require" for SSL [OK]

Hint: Use "sslmode": "require" to enable SSL in extras [OK]

Common Mistakes:

• Using incorrect keys like "ssl_enabled"
• Setting SSL to false or no disables encryption
• Wrong JSON format in extras field

3. Given this Airflow connection extra field: {"sslmode": "verify-full", "sslrootcert": "/path/to/ca.pem"}, what is the effect when Airflow connects to the database?

medium

A. Airflow disables SSL and uses plain text

B. Airflow uses SSL but does not verify the server certificate

C. Airflow connects without encryption

D. Airflow uses SSL and verifies the server certificate against the CA file

Solution

1. Step 1: Understand sslmode "verify-full"

   This mode means SSL is used and the server certificate is fully verified.

2. Step 2: Role of sslrootcert

   The "sslrootcert" points to the CA certificate file used to verify the server's certificate authenticity.

3. Final Answer:

   Airflow uses SSL and verifies the server certificate against the CA file -> Option D

4. Quick Check:

   "verify-full" + CA cert = full SSL verification [OK]

Hint: "verify-full" means SSL with full certificate check [OK]

Common Mistakes:

• Confusing "verify-full" with no verification
• Ignoring the CA certificate path
• Assuming SSL is off with these settings

4. You configured an Airflow connection with {"sslmode": "require"} in the extra field, but Airflow still connects without encryption. What is the most likely cause?

medium

A. The database server does not support SSL connections

B. The Airflow scheduler was not restarted after the change

C. The extra field JSON is missing a closing brace

D. The connection type does not support SSL encryption

Solution

1. Step 1: Airflow connection caching

   Airflow components like the scheduler cache connections in memory. Changes to connections require restarting to load new configuration.

2. Step 2: Verify other causes

   Without restart, the new SSL extra is ignored, using cached unencrypted connection. Server lacking SSL support would fail the connection; malformed JSON or unsupported conn type would error differently.

3. Final Answer:

   The Airflow scheduler was not restarted after the change -> Option B

4. Quick Check:

   Restart Airflow after connection changes [OK]

Hint: Restart Airflow scheduler after connection config changes [OK]

Common Mistakes:

• Assuming Airflow ignores server capabilities
• Not checking server SSL configuration
• Forgetting to restart Airflow after config changes

5. You want to configure an Airflow connection to a PostgreSQL database with full SSL verification using a CA certificate file located at /certs/ca.pem. Which extra field JSON is correct?

hard

A. {"sslmode": "require", "sslrootcert": "/certs/ca.pem"}

B. {"sslmode": "disable", "sslrootcert": "/certs/ca.pem"}

C. {"sslmode": "verify-full", "sslrootcert": "/certs/ca.pem"}

D. {"sslmode": "verify-ca", "sslrootcert": "/certs/ca.pem"}

Solution

1. Step 1: Understand SSL modes for PostgreSQL

   "verify-full" requires SSL and full verification of the server certificate against the CA file.

2. Step 2: Compare other modes

   "require" enables SSL but does not verify the server certificate fully; "disable" turns off SSL; "verify-ca" verifies the CA but not the hostname.

3. Final Answer:

   {"sslmode": "verify-full", "sslrootcert": "/certs/ca.pem"} -> Option C

4. Quick Check:

   Full verification needs "verify-full" mode [OK]

Hint: "verify-full" + CA cert = strongest SSL verification [OK]

Common Mistakes:

• Using "require" instead of "verify-full" for full verification
• Confusing "verify-ca" with full hostname verification
• Disabling SSL accidentally