Variable encryption for secrets in Apache Airflow - Mini Project: Build & Apply

Variable encryption for secrets

📖 Scenario: You are managing an Apache Airflow environment where you need to store sensitive information like API keys securely. Airflow allows you to store variables, but storing secrets in plain text is risky. To protect these secrets, Airflow supports encrypting variables.In this project, you will simulate storing a secret variable in Airflow with encryption enabled, and then retrieve and decrypt it safely.

🎯 Goal: Build a simple Python script that simulates storing an encrypted secret variable and retrieving it decrypted, mimicking Airflow's variable encryption feature.

📋 What You'll Learn

Create a dictionary to simulate Airflow variables storage

Add a configuration variable to represent encryption key

Write a function to encrypt and decrypt variable values

Store an encrypted secret and retrieve it decrypted

💡 Why This Matters

🌍 Real World

In real Airflow deployments, secrets like API keys and passwords must be encrypted to prevent unauthorized access. This project shows the basic idea behind encrypting variables before storing them.

💼 Career

Understanding how to securely manage secrets is essential for DevOps engineers and data engineers working with Airflow or any automation platform that handles sensitive data.

1

Create Airflow variables storage

Create a dictionary called airflow_variables with one entry: the key 'api_key' and the value 'my_secret_api_key'.

Apache Airflow

# Create the airflow_variables dictionary with the specified entry
# Your code here

Hint

Use curly braces to create a dictionary with the key 'api_key' and the value 'my_secret_api_key'.

2

Add encryption key configuration

Create a variable called encryption_key and set it to the string 'simplekey'.

Apache Airflow

airflow_variables = {'api_key': 'my_secret_api_key'}
# Create the encryption_key variable with the value 'simplekey'
# Your code here

Hint

Just assign the string 'simplekey' to the variable encryption_key.

3

Write encryption and decryption functions

Define two functions: encrypt(value, key) and decrypt(value, key). Use a simple Caesar cipher shifting each character by the length of key. For encryption, shift characters forward; for decryption, shift backward.

Apache Airflow

airflow_variables = {'api_key': 'my_secret_api_key'}
encryption_key = 'simplekey'
# Define encrypt(value, key) and decrypt(value, key) functions
# Your code here

Hint

Use ord() and chr() to shift characters by the length of the key.

4

Store encrypted secret and print decrypted value

Encrypt the 'api_key' value from airflow_variables using encrypt and encryption_key. Store it back in airflow_variables['api_key']. Then decrypt it and print the decrypted value.

Apache Airflow

airflow_variables = {'api_key': 'my_secret_api_key'}
encryption_key = 'simplekey'

def encrypt(value, key):
    shift = len(key)
    encrypted = ''.join(chr((ord(c) + shift) % 256) for c in value)
    return encrypted

def decrypt(value, key):
    shift = len(key)
    decrypted = ''.join(chr((ord(c) - shift) % 256) for c in value)
    return decrypted

# Encrypt the api_key, store it, then decrypt and print it
# Your code here

Hint

Use the encrypt function on the original value, store it, then decrypt and print the result.