How to Pull XCom in Airflow: Syntax and Examples

To pull an XCom value in Airflow, use task_instance.xcom_pull(task_ids='task_id', key='key_name') inside your task function. This fetches data pushed by another task identified by task_id and key.

Syntax

The basic syntax to pull an XCom value in Airflow is:

• task_instance.xcom_pull(task_ids='task_id', key='key_name'): Pulls the XCom value from the specified task.
• task_ids: The ID of the task that pushed the XCom.
• key: The key name of the XCom value to retrieve. Defaults to return_value if not specified.

python

value = task_instance.xcom_pull(task_ids='my_task', key='my_key')

Example

This example shows two PythonOperator tasks where the first task pushes an XCom value and the second task pulls it.

python

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime

def push_function(ti):
    ti.xcom_push(key='sample_key', value='Hello from push task')

def pull_function(ti):
    pulled_value = ti.xcom_pull(task_ids='push_task', key='sample_key')
    print(f'Pulled XCom value: {pulled_value}')

default_args = {'start_date': datetime(2024, 1, 1)}

dag = DAG('xcom_example', default_args=default_args, schedule_interval=None)

push_task = PythonOperator(
    task_id='push_task',
    python_callable=push_function,
    dag=dag
)

pull_task = PythonOperator(
    task_id='pull_task',
    python_callable=pull_function,
    dag=dag
)

push_task >> pull_task

Output

Pulled XCom value: Hello from push task

Common Pitfalls

Common mistakes when pulling XComs include:

• Not specifying the correct task_ids that pushed the XCom.
• Using the wrong key or forgetting it when a custom key was used.
• Trying to pull XCom outside of a task context where task_instance is not available.
• Expecting XCom to share data across DAG runs without proper configuration.

Always ensure your pull happens inside a task function with access to task_instance.

python

## Wrong way (missing task_instance context)
def pull_wrong():
    value = task_instance.xcom_pull(task_ids='push_task')  # task_instance undefined

## Right way
from airflow.operators.python import get_current_context
def pull_right():
    ti = get_current_context()['ti']
    value = ti.xcom_pull(task_ids='push_task')

Quick Reference

Remember these tips when working with XCom pulls:

• Use task_instance.xcom_pull() inside task functions.
• Specify task_ids to identify the source task.
• Use key if you pushed with a custom key; otherwise, default is return_value.
• XComs are meant for small data; avoid large payloads.

Key Takeaways

Use task_instance.xcom_pull(task_ids='task_id', key='key_name') inside task functions to get XCom values.

Always specify the correct task_id and key to avoid pulling wrong or no data.

Pull XComs only within task context where task_instance is available.

XComs are for small data sharing between tasks, not for large files or datasets.

Use xcom_push with matching keys to store data for later retrieval.