Connection encryption in Apache Airflow - Mini Project: Build & Apply

Connection encryption in Apache Airflow

📖 Scenario: You are setting up Apache Airflow to securely connect to a PostgreSQL database. To protect sensitive data, you need to configure connection encryption.

🎯 Goal: Configure an Airflow connection with encryption enabled to securely connect to a PostgreSQL database.

📋 What You'll Learn

Create an Airflow connection dictionary with exact parameters

Add an encryption flag to the connection configuration

Use the Airflow CLI command to add the connection with encryption

Print the connection details to verify encryption is enabled

💡 Why This Matters

🌍 Real World

In real projects, encrypting connections protects sensitive data when Airflow talks to databases or services.

💼 Career

Knowing how to configure encrypted connections is essential for DevOps roles managing secure data pipelines.

1

Create Airflow connection dictionary

Create a dictionary called conn_postgres with these exact keys and values: conn_id set to "my_postgres_conn", conn_type set to "postgres", host set to "localhost", login set to "airflow_user", password set to "airflow_pass", and schema set to "airflow_db".

Apache Airflow

# Create the Airflow connection dictionary with exact keys and values
# Your code here

Hint

Use a Python dictionary with the exact keys and values as specified.

2

Add encryption flag to connection dictionary

Add a key extra to the conn_postgres dictionary with the exact value '{"sslmode": "require"}' to enable encryption.

Apache Airflow

conn_postgres = {
    "conn_id": "my_postgres_conn",
    "conn_type": "postgres",
    "host": "localhost",
    "login": "airflow_user",
    "password": "airflow_pass",
    "schema": "airflow_db"
}
# Add the 'extra' key with encryption settings below
# Your code here

Hint

The extra key must be a string containing JSON with "sslmode": "require".

3

Add the connection using Airflow CLI command

Write a string variable called cli_command that contains the exact Airflow CLI command to add the connection with encryption: airflow connections add my_postgres_conn --conn-type postgres --conn-host localhost --conn-login airflow_user --conn-password airflow_pass --conn-schema airflow_db --conn-extra '{"sslmode": "require"}'

Apache Airflow

conn_postgres = {
    "conn_id": "my_postgres_conn",
    "conn_type": "postgres",
    "host": "localhost",
    "login": "airflow_user",
    "password": "airflow_pass",
    "schema": "airflow_db",
    "extra": '{"sslmode": "require"}'
}
# Write the Airflow CLI command string to add the connection below
# Your code here

Hint

Escape double quotes inside the conn-extra JSON string properly.

4

Print the connection dictionary to verify encryption

Write a print statement to display the conn_postgres dictionary.

Apache Airflow

conn_postgres = {
    "conn_id": "my_postgres_conn",
    "conn_type": "postgres",
    "host": "localhost",
    "login": "airflow_user",
    "password": "airflow_pass",
    "schema": "airflow_db",
    "extra": '{"sslmode": "require"}'
}

cli_command = "airflow connections add my_postgres_conn --conn-type postgres --conn-host localhost --conn-login airflow_user --conn-password airflow_pass --conn-schema airflow_db --conn-extra '{\"sslmode\": \"require\"}'"
# Print the connection dictionary below
# Your code here

Hint

Use print(conn_postgres) to show the dictionary.