DAG versioning strategies in Apache Airflow - Mini Project: Build & Apply

DAG Versioning Strategies in Apache Airflow

📖 Scenario: You work as a data engineer managing workflows in Apache Airflow. Your team wants to keep track of different versions of DAGs (Directed Acyclic Graphs) to safely update workflows without breaking existing runs.This project will guide you through creating a simple DAG with versioning information embedded, so you can see how to manage DAG versions clearly.

🎯 Goal: Build a basic Airflow DAG that includes a version number in its ID and a configuration variable for the version. Then, write code to print the DAG version when the DAG runs.

📋 What You'll Learn

Create a DAG dictionary with a fixed DAG ID and version number

Add a configuration variable to hold the DAG version

Write a function to print the DAG version during execution

Print the DAG version as the final output

💡 Why This Matters

🌍 Real World

In real Airflow projects, managing DAG versions helps teams deploy updates safely and track changes over time.

💼 Career

Understanding DAG versioning is important for data engineers and DevOps professionals who maintain reliable workflow pipelines.

1

Create the initial DAG dictionary

Create a dictionary called dag_info with these exact entries: 'dag_id': 'example_dag_v1' and 'version': '1.0'.

Apache Airflow

# Create the dag_info dictionary with dag_id and version
# Your code here

Hint

Use curly braces to create a dictionary with keys 'dag_id' and 'version'.

2

Add a configuration variable for DAG version

Create a variable called dag_version and set it equal to the 'version' value from the dag_info dictionary.

Apache Airflow

dag_info = {'dag_id': 'example_dag_v1', 'version': '1.0'}
# Create dag_version variable from dag_info['version']
# Your code here

Hint

Access the dictionary value using square brackets and the key 'version'.

3

Write a function to print the DAG version

Define a function called print_dag_version that takes no arguments and prints the text "DAG version: {dag_version}" using an f-string.

Apache Airflow

dag_info = {'dag_id': 'example_dag_v1', 'version': '1.0'}
dag_version = dag_info['version']
# Define print_dag_version function to print DAG version
# Your code here

Hint

Use def to define the function and an f-string inside print().

4

Print the DAG version output

Call the function print_dag_version() to display the DAG version.

Apache Airflow

dag_info = {'dag_id': 'example_dag_v1', 'version': '1.0'}
dag_version = dag_info['version']

def print_dag_version():
    print(f"DAG version: {dag_version}")
# Call print_dag_version to show the version
# Your code here

Hint

Simply call the function by its name followed by parentheses.