0
0
AirflowHow-ToBeginner · 4 min read

How to Push XCom in Airflow: Syntax and Examples

In Airflow, you push an XCom value by calling ti.xcom_push(key, value) inside a task function where ti is the task instance. Alternatively, with the TaskFlow API, you can return a value from a task function to push it automatically as an XCom.
📐

Syntax

To push an XCom manually, use ti.xcom_push(key, value) inside your task function. Here, ti is the task instance passed as a parameter. The key is a string identifier for the data, and value is the data you want to share.

With the TaskFlow API, simply return the value from the task function to push it as an XCom automatically.

python
def push_function(ti):
    ti.xcom_push(key='sample_key', value='Hello from XCom')
💻

Example

This example shows two tasks: one pushes an XCom value manually, and the other pulls and prints it. It uses PythonOperator and demonstrates how to share data between tasks.

python
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
from airflow.decorators import task

def push_function(ti):
    ti.xcom_push(key='message', value='Hello Airflow XCom!')

def pull_function(ti):
    message = ti.xcom_pull(key='message', task_ids='push_task')
    print(f'Received message: {message}')

@task
def taskflow_push_task():
    return 'Hello from TaskFlow!'

@task
def taskflow_pull_task(message):
    print(f'TaskFlow received: {message}')

with DAG(dag_id='xcom_example', start_date=days_ago(1), schedule_interval=None) as dag:
    push_task = PythonOperator(
        task_id='push_task',
        python_callable=push_function
    )

    pull_task = PythonOperator(
        task_id='pull_task',
        python_callable=pull_function
    )

    taskflow_push = taskflow_push_task()
    taskflow_pull = taskflow_pull_task(taskflow_push)

    push_task >> pull_task
Output
Received message: Hello Airflow XCom! TaskFlow received: Hello from TaskFlow!
⚠️

Common Pitfalls

  • Not passing ti (task instance) to the Python callable will cause errors when calling ti.xcom_push.
  • Using the wrong key or task_id when pulling XComs will return None.
  • Forgetting that TaskFlow API pushes XCom automatically by returning values, so manual xcom_push is unnecessary there.
python
def wrong_push_function():
    # Missing 'ti' parameter
    ti.xcom_push(key='key', value='value')  # This will fail

def correct_push_function(ti):
    ti.xcom_push(key='key', value='value')  # Correct usage
📊

Quick Reference

ActionCode ExampleDescription
Push XCom manuallyti.xcom_push(key='my_key', value='data')Push data with a key inside a task function
Pull XComti.xcom_pull(key='my_key', task_ids='task_id')Retrieve data pushed by another task
TaskFlow pushdef task(): return 'value'Return value auto-pushed as XCom
TaskFlow pullti.xcom_pull(task_ids='task_id')Pull XCom from TaskFlow task

Key Takeaways

Use ti.xcom_push(key, value) inside a task function to push XCom manually.
With TaskFlow API, simply return a value to push it as XCom automatically.
Always pass the task instance (ti) to your Python callable when using xcom_push or xcom_pull.
Use correct keys and task IDs to pull the right XCom values.
Avoid mixing manual xcom_push with TaskFlow return values for the same data.