Connection encryption in Apache Airflow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is connection encryption in Airflow?

Connection encryption in Airflow means protecting data sent between Airflow and external systems by making it unreadable to outsiders.

beginner

Why should you use connection encryption in Airflow?

To keep sensitive information like passwords and tokens safe from hackers when Airflow talks to databases or APIs.

intermediate

Which protocol is commonly used for encrypting connections in Airflow?

TLS (Transport Layer Security) is commonly used to encrypt connections, ensuring data privacy and integrity.

intermediate

How do you enable SSL/TLS encryption for a database connection in Airflow?

You add SSL parameters in the connection URI or extra field, like sslmode=require for PostgreSQL, to tell Airflow to use encryption.

intermediate

What is the role of the 'extra' field in Airflow connections for encryption?

The 'extra' field lets you add JSON settings like SSL certificates or encryption options to secure the connection.

What does connection encryption protect in Airflow?

A Data sent between Airflow and external systems

B Only Airflow's internal logs

C Airflow's UI layout

D Airflow's scheduler timing

Connection encryption protects data sent between Airflow and external systems from being read by unauthorized users.

Which protocol is most commonly used to encrypt Airflow connections?

A FTP

B SMTP

C HTTP

D TLS

TLS (Transport Layer Security) is the standard protocol for encrypting connections to keep data private and secure.

Where do you add SSL parameters in Airflow to enable encryption for a connection?

A In the connection URI or 'extra' field

B In the Airflow scheduler logs

C In the DAG file

D In the Airflow webserver config

SSL parameters are added in the connection URI or the 'extra' JSON field to enable encryption for that connection.

What does the 'sslmode=require' option do in a PostgreSQL connection in Airflow?

A Sets the connection timeout

B Disables encryption

C Requires SSL encryption for the connection

D Enables debug logging

'sslmode=require' forces the connection to use SSL encryption, protecting data in transit.

Why is encrypting Airflow connections important?

A To speed up data transfer

B To protect sensitive data from being intercepted

C To reduce CPU usage

D To improve Airflow UI appearance

Encrypting connections protects sensitive data like passwords and tokens from being intercepted by attackers.

Explain how to enable connection encryption for a database in Airflow.

Think about where Airflow stores connection details and how to tell it to use encryption.

Why is connection encryption critical when Airflow communicates with external systems?

Consider what could happen if data is sent without protection.

Practice

1. What is the main purpose of enabling connection encryption in Airflow?

easy

A. To protect data exchanged between Airflow and external systems

B. To speed up the connection between Airflow and databases

C. To reduce the size of data transferred over the network

D. To allow Airflow to connect without authentication

Solution

1. Step 1: Understand connection encryption

   Encryption secures data by making it unreadable to outsiders during transfer.

2. Step 2: Apply to Airflow connections

   Airflow uses encryption to protect sensitive data when communicating with external systems.

3. Final Answer:

   To protect data exchanged between Airflow and external systems -> Option A

4. Quick Check:

   Encryption protects data [OK]

Hint: Encryption means data is safe from outsiders [OK]

Common Mistakes:

• Thinking encryption speeds up connections
• Confusing encryption with data compression
• Assuming encryption removes authentication

2. Which of the following is the correct way to specify SSL encryption in an Airflow connection's extra field?

easy

A. {"sslmode": "require"}

B. {"ssl": "false"}

C. {"encrypt": "no"}

D. {"ssl_enabled": "off"}

Solution

1. Step 1: Identify correct SSL option key

   The standard key for SSL mode in Airflow connection extras is "sslmode" with values like "require".

2. Step 2: Check option values

   "require" enables SSL, while other options like "false", "no", or "off" disable encryption.

3. Final Answer:

   {"sslmode": "require"} -> Option A

4. Quick Check:

   Use "sslmode": "require" for SSL [OK]

Hint: Use "sslmode": "require" to enable SSL in extras [OK]

Common Mistakes:

• Using incorrect keys like "ssl_enabled"
• Setting SSL to false or no disables encryption
• Wrong JSON format in extras field

3. Given this Airflow connection extra field: {"sslmode": "verify-full", "sslrootcert": "/path/to/ca.pem"}, what is the effect when Airflow connects to the database?

medium

A. Airflow disables SSL and uses plain text

B. Airflow uses SSL but does not verify the server certificate

C. Airflow connects without encryption

D. Airflow uses SSL and verifies the server certificate against the CA file

Solution

1. Step 1: Understand sslmode "verify-full"

   This mode means SSL is used and the server certificate is fully verified.

2. Step 2: Role of sslrootcert

   The "sslrootcert" points to the CA certificate file used to verify the server's certificate authenticity.

3. Final Answer:

   Airflow uses SSL and verifies the server certificate against the CA file -> Option D

4. Quick Check:

   "verify-full" + CA cert = full SSL verification [OK]

Hint: "verify-full" means SSL with full certificate check [OK]

Common Mistakes:

• Confusing "verify-full" with no verification
• Ignoring the CA certificate path
• Assuming SSL is off with these settings

4. You configured an Airflow connection with {"sslmode": "require"} in the extra field, but Airflow still connects without encryption. What is the most likely cause?

medium

A. The database server does not support SSL connections

B. The Airflow scheduler was not restarted after the change

C. The extra field JSON is missing a closing brace

D. The connection type does not support SSL encryption

Solution

1. Step 1: Airflow connection caching

   Airflow components like the scheduler cache connections in memory. Changes to connections require restarting to load new configuration.

2. Step 2: Verify other causes

   Without restart, the new SSL extra is ignored, using cached unencrypted connection. Server lacking SSL support would fail the connection; malformed JSON or unsupported conn type would error differently.

3. Final Answer:

   The Airflow scheduler was not restarted after the change -> Option B

4. Quick Check:

   Restart Airflow after connection changes [OK]

Hint: Restart Airflow scheduler after connection config changes [OK]

Common Mistakes:

• Assuming Airflow ignores server capabilities
• Not checking server SSL configuration
• Forgetting to restart Airflow after config changes

5. You want to configure an Airflow connection to a PostgreSQL database with full SSL verification using a CA certificate file located at /certs/ca.pem. Which extra field JSON is correct?

hard

A. {"sslmode": "require", "sslrootcert": "/certs/ca.pem"}

B. {"sslmode": "disable", "sslrootcert": "/certs/ca.pem"}

C. {"sslmode": "verify-full", "sslrootcert": "/certs/ca.pem"}

D. {"sslmode": "verify-ca", "sslrootcert": "/certs/ca.pem"}

Solution

1. Step 1: Understand SSL modes for PostgreSQL

   "verify-full" requires SSL and full verification of the server certificate against the CA file.

2. Step 2: Compare other modes

   "require" enables SSL but does not verify the server certificate fully; "disable" turns off SSL; "verify-ca" verifies the CA but not the hostname.

3. Final Answer:

   {"sslmode": "verify-full", "sslrootcert": "/certs/ca.pem"} -> Option C

4. Quick Check:

   Full verification needs "verify-full" mode [OK]

Hint: "verify-full" + CA cert = strongest SSL verification [OK]

Common Mistakes:

• Using "require" instead of "verify-full" for full verification
• Confusing "verify-ca" with full hostname verification
• Disabling SSL accidentally