XCom with return values in Apache Airflow - Mini Project: Build & Apply

XCom with return values

📖 Scenario: You are working with Apache Airflow to automate tasks. You want to pass data between tasks using XComs, which allow tasks to share small pieces of information.Imagine you have two tasks: one that generates a number, and another that uses that number to print a message.

🎯 Goal: Build a simple Airflow DAG with two Python tasks. The first task returns a value using XCom. The second task reads that value and prints it.

📋 What You'll Learn

Create a Python function called generate_number that returns the integer 42.

Create a Python function called print_number that accepts ti as a parameter and pulls the XCom value from generate_number task.

Create a DAG named xcom_return_value_dag with two tasks: task_generate and task_print.

Set task_generate to run generate_number and task_print to run print_number.

Set task_generate to run before task_print.

Print the pulled XCom value inside print_number.

💡 Why This Matters

🌍 Real World

In real Airflow workflows, tasks often need to share data like IDs, statuses, or results. XComs let you pass this data easily between tasks.

💼 Career

Understanding XComs is essential for Airflow users and DevOps engineers who automate workflows and need tasks to communicate with each other.

1

Create the generate_number function

Write a Python function called generate_number that returns the integer 42.

Apache Airflow

# Your code here

Need a hint?

Define a function with def and use return 42 inside it.

2

Create the print_number function to pull XCom

Write a Python function called print_number that accepts a parameter ti. Inside it, pull the XCom value from the task task_generate using ti.xcom_pull(task_ids='task_generate') and store it in a variable called number.

Apache Airflow

def generate_number():
    return 42

# Your code here

Need a hint?

Use ti.xcom_pull(task_ids='task_generate') to get the value returned by generate_number.

3

Create the DAG and tasks

Import DAG and PythonOperator from Airflow. Create a DAG named xcom_return_value_dag with start_date as days_ago(1) and schedule_interval as None. Inside the DAG, create two tasks: task_generate running generate_number and task_print running print_number. Set task_generate to run before task_print.

Apache Airflow

def generate_number():
    return 42

def print_number(ti):
    number = ti.xcom_pull(task_ids='task_generate')

# Your code here

Need a hint?

Use PythonOperator to run your functions as tasks inside the DAG context.

4

Print the pulled XCom value in print_number

Add a print statement inside the print_number function to display the variable number.

Apache Airflow

from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago

def generate_number():
    return 42

def print_number(ti):
    number = ti.xcom_pull(task_ids='task_generate')
    # Your code here

with DAG(
    'xcom_return_value_dag',
    start_date=days_ago(1),
    schedule_interval=None
) as dag:
    task_generate = PythonOperator(
        task_id='task_generate',
        python_callable=generate_number
    )

    task_print = PythonOperator(
        task_id='task_print',
        python_callable=print_number
    )

    task_generate >> task_print

Need a hint?

Use print(number) to show the value pulled from XCom.