Cost optimization for cloud resources in Apache Airflow - Mini Project: Build & Apply

Cost optimization for cloud resources

📖 Scenario: You work as a cloud engineer managing multiple cloud resources. Your goal is to identify resources that cost more than a certain threshold and mark them for review to optimize costs.

🎯 Goal: Build an Airflow DAG that loads a list of cloud resources with their costs, sets a cost threshold, filters resources exceeding this threshold, and prints the list of expensive resources for review.

📋 What You'll Learn

Create a dictionary of cloud resources with their monthly costs

Set a cost threshold variable

Filter resources with costs above the threshold using a comprehension

Print the filtered list of expensive resources

💡 Why This Matters

🌍 Real World

Cloud engineers often need to monitor and optimize cloud resource costs to avoid overspending.

💼 Career

This project teaches how to filter and identify costly cloud resources programmatically, a key skill in cloud cost management roles.

1

Create the cloud resources cost dictionary

Create a dictionary called cloud_resources with these exact entries: 'VM1': 120, 'DB1': 80, 'Storage1': 45, 'VM2': 200, 'Cache1': 30 representing resource names and their monthly costs.

Apache Airflow

# Create the cloud_resources dictionary with exact entries
# Your code here

Hint

Use curly braces to create a dictionary with keys as resource names and values as costs.

2

Set the cost threshold

Create a variable called cost_threshold and set it to 100 to represent the monthly cost limit for resources.

Apache Airflow

cloud_resources = {'VM1': 120, 'DB1': 80, 'Storage1': 45, 'VM2': 200, 'Cache1': 30}
# Set cost_threshold to 100
# Your code here

Hint

Just assign the number 100 to the variable cost_threshold.

3

Filter expensive resources

Use a dictionary comprehension to create a new dictionary called expensive_resources that contains only the resources from cloud_resources with costs greater than cost_threshold. Use name and cost as the loop variables.

Apache Airflow

cloud_resources = {'VM1': 120, 'DB1': 80, 'Storage1': 45, 'VM2': 200, 'Cache1': 30}
cost_threshold = 100
# Create expensive_resources dictionary with resources costing more than cost_threshold
# Your code here

Hint

Use a dictionary comprehension with for name, cost in cloud_resources.items() and an if condition.

4

Print the expensive resources

Write a print statement to display the expensive_resources dictionary.

Apache Airflow

cloud_resources = {'VM1': 120, 'DB1': 80, 'Storage1': 45, 'VM2': 200, 'Cache1': 30}
cost_threshold = 100
expensive_resources = {name: cost for name, cost in cloud_resources.items() if cost > cost_threshold}
# Print the expensive_resources dictionary
# Your code here

Hint

Use print(expensive_resources) to show the filtered dictionary.