XCom with return values in Apache Airflow - Step-by-Step Execution

Process Flow - XCom with return values

Task A runs

↓

Task A returns value

↓

Airflow stores value in XCom

↓

Task B starts

↓

Task B pulls value from XCom

↓

Task B uses value

Task A runs and returns a value, which Airflow saves in XCom. Task B then pulls this value from XCom to use it.

Execution Sample

Apache Airflow

def task_a():
    return 'hello from A'

def task_b(ti):
    msg = ti.xcom_pull(task_ids='task_a')
    print(msg)

Task A returns a string; Task B pulls that string from XCom and prints it.

Process Table

Step	Action	Task	XCom State	Output/Result
1	Run task_a()	task_a	XCom empty	task_a returns 'hello from A'
2	Store return value in XCom	task_a	XCom stores {'task_a': 'hello from A'}	Value saved
3	Start task_b()	task_b	XCom has {'task_a': 'hello from A'}	task_b starts
4	Pull value from XCom	task_b	XCom unchanged	Pulled 'hello from A'
5	Print pulled value	task_b	XCom unchanged	Output: hello from A
6	task_b ends	task_b	XCom unchanged	Execution complete

💡 task_b finishes after pulling and printing the value from XCom

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 4	Final
XCom	empty	empty	{'task_a': 'hello from A'}	{'task_a': 'hello from A'}	{'task_a': 'hello from A'}
msg (in task_b)	undefined	undefined	undefined	'hello from A'	'hello from A'

Key Moments - 2 Insights

Why does task_b need to pull the value from XCom instead of receiving it directly?

Because Airflow tasks run independently, the return value from task_a is stored in XCom. task_b must explicitly pull it from XCom, as shown in step 4 of the execution_table.

What happens if task_a does not return anything?

If task_a returns nothing, XCom stores None or no value for task_a. Then, when task_b pulls from XCom (step 4), it gets None, so no useful data is passed.

Visual Quiz - 3 Questions

Look at the execution_table at step 2. What is stored in XCom?

A The print output from task_b

B An empty dictionary

C The string 'hello from A' under task_a key

D Nothing is stored yet

💡 Hint

Check the 'XCom State' column at step 2 in execution_table

At which step does task_b retrieve the value from XCom?

A Step 4

B Step 1

C Step 3

D Step 5

💡 Hint

Look for 'Pull value from XCom' action in execution_table

If task_a returned None, what would 'msg' be after step 4?

A 'hello from A'

B None

C An error

D Empty string

💡 Hint

Refer to variable_tracker for 'msg' values and key_moments explanation about no return

Concept Snapshot

XCom lets Airflow tasks share data.
Task returns a value, Airflow stores it in XCom.
Other tasks pull this value by task_id.
Use ti.xcom_pull(task_ids='task_id') to get it.
Useful for passing small data between tasks.

Full Transcript

In Airflow, tasks run separately. When a task returns a value, Airflow saves it in a special place called XCom. Another task can then ask XCom for that value using the task's ID. This way, tasks can share information even though they run independently. For example, task_a returns 'hello from A', Airflow stores it, and task_b pulls and prints it. If task_a returns nothing, task_b gets None. This process helps tasks communicate small pieces of data.