Why XCom enables task communication in Apache Airflow - Visual Breakdown

Process Flow - Why XCom enables task communication

Task A runs

↓

Task A pushes XCom

↓

XCom stores data

↓

Task B starts

↓

Task B pulls XCom

↓

Task B uses data

↓

Task B completes

Task A runs and pushes data to XCom, which stores it. Task B later pulls this data from XCom to use it, enabling communication between tasks.

Execution Sample

Apache Airflow

def task_a(ti):
    ti.xcom_push(key='result', value='data from A')

def task_b(ti):
    data = ti.xcom_pull(key='result', task_ids='task_a')
    print(data)

Task A pushes data to XCom; Task B pulls and prints it.

Process Table

Step	Action	XCom State	Output
1	Task A runs	{}	No output
2	Task A pushes XCom with key='result', value='data from A'	{"result": "data from A"}	No output
3	Task B starts	{"result": "data from A"}	No output
4	Task B pulls XCom with key='result' from task_a	{"result": "data from A"}	"data from A"
5	Task B prints pulled data	{"result": "data from A"}	data from A
6	Task B completes	{"result": "data from A"}	No output

💡 Task B completes after successfully pulling and using data from XCom

Status Tracker

Variable	Start	After Step 2	After Step 4	Final
XCom	{}	{"result": "data from A"}	{"result": "data from A"}	{"result": "data from A"}
data (in task_b)	N/A	N/A	"data from A"	"data from A"

Key Moments - 2 Insights

Why does Task B need to specify task_ids='task_a' when pulling from XCom?

Because XCom stores data per task, Task B must specify which task's data it wants to pull. See execution_table step 4 where task_b pulls with task_ids='task_a'.

Can Task A push multiple values with the same key to XCom?

No, each key-task pair stores one value. Pushing again with the same key overwrites the previous value. This is why in execution_table step 2, the key 'result' holds one value.

Visual Quiz - 3 Questions

Look at the execution table, what is the XCom state after Task A pushes data?

A {"task_b": "data from A"}

B {}

C {"result": "data from A"}

D null

💡 Hint

Check execution_table row 2 under XCom State column

At which step does Task B retrieve the data from XCom?

A Step 4

B Step 5

C Step 3

D Step 6

💡 Hint

Look at execution_table rows and find when XCom pull happens

If Task A pushed data with a different key, what would Task B need to change?

A Change the task_ids parameter

B Change the key in xcom_pull to match the new key

C No change needed

D Change the print statement

💡 Hint

Refer to execution_table step 4 where key='result' is used to pull data

Concept Snapshot

XCom allows tasks to share data by pushing and pulling key-value pairs.
Task A pushes data with ti.xcom_push(key, value).
Task B pulls data with ti.xcom_pull(key, task_ids).
XCom stores data per task and key.
This enables communication between tasks in Airflow DAGs.

Full Transcript

In Airflow, tasks can communicate by using XComs. Task A runs and pushes data to XCom with a key and value. This data is stored in XCom associated with Task A. Later, Task B starts and pulls the data from XCom by specifying the key and the task id of Task A. Task B then uses this data, for example by printing it. This process allows tasks to share information and coordinate their work. The execution table shows each step: Task A pushing data, XCom storing it, Task B pulling it, and finally Task B using it. Variables like XCom and data in Task B change accordingly. Key points include that Task B must specify which task's data to pull, and that each key-task pair holds one value. This simple push-pull mechanism enables task communication in Airflow.