TaskFlow API for cleaner XCom in Apache Airflow - Mini Project: Build & Apply

TaskFlow API for Cleaner XCom in Airflow

📖 Scenario: You are working with Apache Airflow to automate a simple data processing workflow. You want to use the TaskFlow API to pass data between tasks cleanly without manually handling XComs.

🎯 Goal: Build a simple Airflow DAG using the TaskFlow API where one task returns a value and the next task receives it as input automatically.

📋 What You'll Learn

Create a DAG with two tasks using the TaskFlow API

First task returns a string value

Second task receives the returned value as a parameter

Print the received value in the second task

💡 Why This Matters

🌍 Real World

In real projects, Airflow workflows often need to pass data between tasks. The TaskFlow API makes this easier and cleaner by using Python functions and automatic XCom handling.

💼 Career

Understanding the TaskFlow API is important for data engineers and DevOps professionals who build and maintain reliable, readable, and maintainable data pipelines using Airflow.

1

Create the DAG and first task

Create a DAG called taskflow_dag with start_date set to days_ago(1) and schedule_interval set to None. Then create a Python function called extract_data that returns the string 'Hello from TaskFlow'. Use the @task decorator from airflow.decorators on this function.

Apache Airflow

from airflow import DAG
from airflow.utils.dates import days_ago
from airflow.decorators import task

# Create DAG and define extract_data task
# Your code here

Need a hint?

Use with DAG(...): to create the DAG context. Decorate the function with @task and return the exact string.

2

Create the second task to receive data

Inside the same DAG context, create a Python function called print_data decorated with @task that takes one parameter called data. Inside the function, write a print statement to display the data parameter.

Apache Airflow

from airflow import DAG
from airflow.utils.dates import days_ago
from airflow.decorators import task

with DAG('taskflow_dag', start_date=days_ago(1), schedule_interval=None) as dag:
    @task
    def extract_data():
        return 'Hello from TaskFlow'

    # Create print_data task that accepts data parameter
    # Your code here

Need a hint?

Define print_data with one parameter data and print it inside the function.

3

Link the tasks using TaskFlow API

Call the extract_data task and assign its result to a variable called extracted. Then call the print_data task passing extracted as the argument. This will link the tasks and pass data automatically.

Apache Airflow

from airflow import DAG
from airflow.utils.dates import days_ago
from airflow.decorators import task

with DAG('taskflow_dag', start_date=days_ago(1), schedule_interval=None) as dag:
    @task
    def extract_data():
        return 'Hello from TaskFlow'

    @task
    def print_data(data):
        print(data)

    # Call extract_data and assign to extracted
    # Call print_data with extracted
    # Your code here

Need a hint?

Assign the result of extract_data() to extracted and pass it to print_data(extracted).

4

Run the DAG and check output

Run the DAG and observe the output of the print_data task. The output should be exactly Hello from TaskFlow printed in the task logs.

Apache Airflow

from airflow import DAG
from airflow.utils.dates import days_ago
from airflow.decorators import task

with DAG('taskflow_dag', start_date=days_ago(1), schedule_interval=None) as dag:
    @task
    def extract_data():
        return 'Hello from TaskFlow'

    @task
    def print_data(data):
        print(data)

    extracted = extract_data()
    print_data(extracted)

# Run the DAG and check output

Need a hint?

Check the logs of the print_data task after running the DAG to see the printed message.