SimpleHttpOperator for API calls in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

What is the output of this Airflow task log snippet?

Given this SimpleHttpOperator task in Airflow, what will the log show after a successful GET request?

Apache Airflow

from airflow import DAG
from airflow.providers.http.operators.http import SimpleHttpOperator
from datetime import datetime

with DAG('test_dag', start_date=datetime(2023,1,1), schedule_interval='@once') as dag:
    task = SimpleHttpOperator(
        task_id='get_ip',
        method='GET',
        http_conn_id='httpbin_default',
        endpoint='ip'
    )

# Assume httpbin_default points to https://httpbin.org
# The endpoint /ip returns JSON with your IP address

A INFO - Response: {"origin": "123.123.123.123"}

B ERROR - Connection refused

C INFO - Response: <html>...</html>

D INFO - Response: 404 Not Found

💡 Hint

The endpoint /ip on httpbin.org returns your IP in JSON format.

The SimpleHttpOperator makes a GET request to https://httpbin.org/ip which returns JSON with the client's IP address under the 'origin' key.

❓ Configuration

intermediate

Which http_conn_id configuration is correct for SimpleHttpOperator?

You want to configure Airflow's connection to call https://api.example.com with token authentication. Which connection setup is correct for http_conn_id?

A Conn Type: HTTP, Host: https://api.example.com, Extra: {"Authorization": "Bearer TOKEN123"}

B Conn Type: HTTP, Host: api.example.com, Extra: {"Authorization": "Bearer TOKEN123"}

C Conn Type: FTP, Host: https://api.example.com, Extra: {"Authorization": "Bearer TOKEN123"}

D Conn Type: HTTP, Host: https://api.example.com, Extra: {"auth_type": "token", "token": "TOKEN123"}

💡 Hint

The Host should not include the protocol (https://).

In Airflow connections, the Host field should be the domain without protocol. The authorization header can be passed in the Extra field as JSON.

🔀 Workflow

advanced

What happens if SimpleHttpOperator receives a 500 HTTP response?

In an Airflow DAG, a SimpleHttpOperator task calls an API endpoint that returns HTTP 500 error. What is the expected behavior?

A The task succeeds but logs the 500 error as a warning.

B The task crashes Airflow scheduler.

C The task ignores the error and continues to downstream tasks.

D The task fails and Airflow retries it if retries are configured.

💡 Hint

HTTP 500 is a server error and SimpleHttpOperator treats non-2xx responses as failures.

SimpleHttpOperator raises an exception on non-2xx HTTP responses, causing the task to fail and trigger retries if configured.

❓ Troubleshoot

advanced

Why does SimpleHttpOperator fail with 'Connection refused' error?

You configured SimpleHttpOperator with http_conn_id pointing to 'my_api'. The task fails with 'Connection refused'. What is the most likely cause?

A The Airflow scheduler is not running.

B The HTTP method is not supported by SimpleHttpOperator.

C The host in 'my_api' connection is incorrect or the server is down.

D The DAG file has syntax errors.

💡 Hint

Connection refused means the network connection to the server failed.

Connection refused usually means the host or port is unreachable, often due to wrong host, server down, or firewall blocking.

✅ Best Practice

expert

How to securely pass sensitive API tokens to SimpleHttpOperator?

You need to call a private API with a token using SimpleHttpOperator. What is the best practice to handle the token securely?

A Store the token in Airflow Connections Extra field and reference http_conn_id in the operator.

B Hardcode the token directly in the DAG Python file in the operator's headers.

C Pass the token as a plain text environment variable and read it in the DAG.

D Include the token in the endpoint URL as a query parameter.

💡 Hint

Airflow Connections are designed to store credentials securely.

Storing tokens in Airflow Connections keeps secrets out of code and allows secure management with Airflow's UI or environment variables.

Practice

1. What is the main purpose of the SimpleHttpOperator in Airflow?

easy

A. To schedule Airflow DAGs

B. To manage database connections

C. To make HTTP API calls within Airflow tasks

D. To send emails from Airflow

Solution

1. Step 1: Understand the role of SimpleHttpOperator

   The SimpleHttpOperator is designed to perform HTTP requests like GET or POST inside Airflow tasks.

2. Step 2: Differentiate from other Airflow features

   Scheduling DAGs, managing databases, or sending emails are handled by other operators, not SimpleHttpOperator.

3. Final Answer:

   To make HTTP API calls within Airflow tasks -> Option C

4. Quick Check:

   SimpleHttpOperator = API calls [OK]

Hint: SimpleHttpOperator = HTTP requests in Airflow tasks [OK]

Common Mistakes:

• Confusing it with scheduling or email operators
• Thinking it manages database connections
• Assuming it triggers DAG runs

2. Which of the following is the correct way to create a SimpleHttpOperator task to perform a GET request to the endpoint '/status' using connection ID 'my_api'?

easy

A. SimpleHttpOperator(task_id='check_status', conn_id='my_api', endpoint='/status', method='POST')

B. SimpleHttpOperator(task_id='check_status', connection_id='my_api', url='/status', method='GET')

C. SimpleHttpOperator(task_id='check_status', http_conn_id='my_api', endpoint='/status', http_method='GET')

D. SimpleHttpOperator(task_id='check_status', http_conn_id='my_api', endpoint='/status', method='GET')

Solution

1. Step 1: Check parameter names for SimpleHttpOperator

   The correct parameters include task_id, http_conn_id, endpoint, and method.

2. Step 2: Validate method and connection ID usage

   SimpleHttpOperator(task_id='check_status', http_conn_id='my_api', endpoint='/status', method='GET') uses http_conn_id and method='GET', which are correct. Other options use wrong parameter names or HTTP methods.

3. Final Answer:

   SimpleHttpOperator(task_id='check_status', http_conn_id='my_api', endpoint='/status', method='GET') -> Option D

4. Quick Check:

   Correct params = SimpleHttpOperator(task_id='check_status', http_conn_id='my_api', endpoint='/status', method='GET') [OK]

Hint: Use http_conn_id and method='GET' for GET requests [OK]

Common Mistakes:

• Using wrong parameter names like connection_id or url
• Confusing method with http_method
• Setting wrong HTTP method for GET request

3. Given this Airflow task code snippet using SimpleHttpOperator:

task = SimpleHttpOperator(
    task_id='get_data',
    http_conn_id='api_conn',
    endpoint='data',
    method='GET'
)

What will be the HTTP method used when this task runs?

medium

A. POST

B. GET

C. PUT

D. DELETE

Solution

1. Step 1: Identify the method parameter in the code

   The code sets method='GET', which specifies the HTTP method to use.

2. Step 2: Confirm the HTTP method used

   Since method='GET', the task will perform a GET request.

3. Final Answer:

   GET -> Option B

4. Quick Check:

   method='GET' means GET request [OK]

Hint: Check the method parameter value for HTTP method [OK]

Common Mistakes:

• Assuming default method is POST
• Confusing method with endpoint
• Ignoring the method parameter

4. You wrote this SimpleHttpOperator task but it fails with an error about missing connection:

task = SimpleHttpOperator(
    task_id='call_api',
    http_conn_id='missing_conn',
    endpoint='info',
    method='GET'
)

What is the most likely cause and fix?

medium

A. The connection ID 'missing_conn' is not defined in Airflow connections; add it in UI

B. The endpoint 'info' is invalid; change it to '/'

C. The method should be lowercase 'get' instead of 'GET'

D. SimpleHttpOperator does not support GET method; use POST instead

Solution

1. Step 1: Understand the error about missing connection

   The error indicates Airflow cannot find the connection ID 'missing_conn' in its stored connections.

2. Step 2: Fix by adding the connection in Airflow UI or CLI

   To fix, define 'missing_conn' with the correct API base URL in Airflow connections before running the task.

3. Final Answer:

   The connection ID 'missing_conn' is not defined in Airflow connections; add it in UI -> Option A

4. Quick Check:

   Missing connection ID = add connection [OK]

Hint: Check Airflow connections for your http_conn_id [OK]

Common Mistakes:

• Assuming endpoint name causes connection error
• Changing HTTP method case unnecessarily
• Thinking GET is unsupported by SimpleHttpOperator

5. You want to use SimpleHttpOperator to POST JSON data {"name": "Alice"} to endpoint '/users' using connection 'user_api'. Which is the correct way to do this?

hard

A. SimpleHttpOperator(task_id='post_user', http_conn_id='user_api', endpoint='/users', method='POST', data='{"name": "Alice"}', headers={'Content-Type': 'application/json'})

B. SimpleHttpOperator(task_id='post_user', http_conn_id='user_api', endpoint='/users', method='POST', json={'name': 'Alice'})

C. SimpleHttpOperator(task_id='post_user', http_conn_id='user_api', endpoint='/users', method='POST', data={'name': 'Alice'})

D. SimpleHttpOperator(task_id='post_user', http_conn_id='user_api', endpoint='/users', method='POST', body='{"name": "Alice"}')

Solution

1. Step 1: Identify how to send JSON data with SimpleHttpOperator

   SimpleHttpOperator uses the data parameter for request body and headers to specify content type.

2. Step 2: Confirm correct JSON string and headers usage

   SimpleHttpOperator(task_id='post_user', http_conn_id='user_api', endpoint='/users', method='POST', data='{"name": "Alice"}', headers={'Content-Type': 'application/json'}) sends JSON as a string in data and sets Content-Type header to 'application/json', which is correct.

3. Final Answer:

   SimpleHttpOperator(task_id='post_user', http_conn_id='user_api', endpoint='/users', method='POST', data='{"name": "Alice"}', headers={'Content-Type': 'application/json'}) -> Option A

4. Quick Check:

   Use data string + JSON header for POST JSON [OK]

Hint: Send JSON as string in data with JSON content-type header [OK]

Common Mistakes:

• Using json parameter which SimpleHttpOperator does not support
• Passing data as dict without converting to string
• Using wrong parameter name like body