SimpleHttpOperator in Airflow: What It Is and How to Use It

SimpleHttpOperator in Airflow is a task operator that lets you send HTTP requests as part of your workflow. It helps automate calling web services or APIs by making GET, POST, or other HTTP calls directly from your Airflow DAGs.

How It Works

SimpleHttpOperator works like a messenger that sends a letter (HTTP request) to a web server and waits for a reply. You tell it the web address (URL), the type of request (like GET or POST), and any extra details like headers or data. Then it sends the request and captures the response.

Think of it as ordering food by phone: you call the restaurant (the web server), place your order (the HTTP request), and wait for confirmation or delivery details (the response). This operator automates that call inside your Airflow workflow, so you don’t have to do it manually.

It uses Airflow’s connection system to store the server details securely, so you only need to reference the connection name in your task. This keeps your credentials safe and your code clean.

Example

This example shows how to use SimpleHttpOperator to send a GET request to a public API and print the response.

python

from airflow import DAG
from airflow.providers.http.operators.http import SimpleHttpOperator
from airflow.utils.dates import days_ago

with DAG(dag_id='http_example', start_date=days_ago(1), schedule_interval='@once') as dag:
    get_ip = SimpleHttpOperator(
        task_id='get_ip',
        http_conn_id='http_default',
        endpoint='ip',
        method='GET',
        response_filter=lambda response: response.text,
        log_response=True
    )

Output

INFO - Response: {"origin": "123.123.123.123"}

When to Use

Use SimpleHttpOperator when your workflow needs to interact with web services or APIs. For example:

• Triggering external systems or services via HTTP calls.
• Fetching data from REST APIs to use later in your workflow.
• Sending notifications or alerts through webhooks.
• Integrating with cloud services that expose HTTP endpoints.

It is best for simple HTTP requests without complex authentication or data processing. For more advanced needs, consider custom operators or hooks.

Key Points

• SimpleHttpOperator sends HTTP requests as Airflow tasks.
• Supports GET, POST, PUT, DELETE methods.
• Uses Airflow connections to manage server URLs and credentials.
• Can process responses with a filter function.
• Ideal for simple API calls within workflows.

Key Takeaways

SimpleHttpOperator automates HTTP requests inside Airflow workflows.

It uses Airflow connections to securely manage API endpoints and credentials.

Best suited for simple GET or POST calls to external web services.

You can log and filter responses easily with built-in options.

Use it to integrate APIs, trigger webhooks, or fetch data in DAGs.