Why access control protects sensitive pipelines in Apache Airflow - See It in Action

Why Access Control Protects Sensitive Pipelines

📖 Scenario: You are managing data workflows using Apache Airflow. Some pipelines handle sensitive data and must be protected from unauthorized access. You want to understand how access control helps keep these pipelines safe.

🎯 Goal: Build a simple example showing how to define a list of pipelines, set an access control list (ACL) for users, filter pipelines based on user permissions, and display only the pipelines a user can access.

📋 What You'll Learn

Create a dictionary called pipelines with exact pipeline names and descriptions

Create a list called user_acl containing exact pipeline names the user can access

Use a dictionary comprehension called accessible_pipelines to filter pipelines by user_acl

Print the accessible_pipelines dictionary to show pipelines the user is allowed to see

💡 Why This Matters

🌍 Real World

In real companies, sensitive data pipelines must be protected so only authorized users can see or run them. Access control helps prevent data leaks and mistakes.

💼 Career

Understanding access control is key for DevOps and data engineers managing workflow tools like Airflow to keep data safe and comply with regulations.

1

Create the pipelines dictionary

Create a dictionary called pipelines with these exact entries: 'daily_sales': 'Process daily sales data', 'user_activity': 'Analyze user activity logs', 'payment_processing': 'Handle payment transactions', 'email_campaign': 'Manage email marketing campaigns'

Apache Airflow

# Create the pipelines dictionary with exact names and descriptions
# Your code here

Hint

Use curly braces {} to create a dictionary with the exact keys and values given.

2

Define the user access control list

Create a list called user_acl containing these exact pipeline names: 'daily_sales' and 'payment_processing'

Apache Airflow

pipelines = {
    'daily_sales': 'Process daily sales data',
    'user_activity': 'Analyze user activity logs',
    'payment_processing': 'Handle payment transactions',
    'email_campaign': 'Manage email marketing campaigns'
}
# Create the user_acl list with exact pipeline names
# Your code here

Hint

Use square brackets [] to create a list with the exact pipeline names given.

3

Filter pipelines by user access

Use a dictionary comprehension called accessible_pipelines to include only the pipelines from pipelines whose keys are in the user_acl list

Apache Airflow

pipelines = {
    'daily_sales': 'Process daily sales data',
    'user_activity': 'Analyze user activity logs',
    'payment_processing': 'Handle payment transactions',
    'email_campaign': 'Manage email marketing campaigns'
}

user_acl = ['daily_sales', 'payment_processing']
# Create accessible_pipelines dictionary with comprehension filtering keys in user_acl
# Your code here

Hint

Use {key: desc for key, desc in pipelines.items() if key in user_acl} to filter the dictionary.

4

Display accessible pipelines

Write a print statement to display the accessible_pipelines dictionary

Apache Airflow

pipelines = {
    'daily_sales': 'Process daily sales data',
    'user_activity': 'Analyze user activity logs',
    'payment_processing': 'Handle payment transactions',
    'email_campaign': 'Manage email marketing campaigns'
}

user_acl = ['daily_sales', 'payment_processing']

accessible_pipelines = {key: desc for key, desc in pipelines.items() if key in user_acl}
# Print the accessible_pipelines dictionary
# Your code here

Hint

Use print(accessible_pipelines) to show the filtered pipelines.