PythonOperator for custom logic in Apache Airflow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of the PythonOperator in Airflow?

PythonOperator lets you run your own Python functions as tasks inside an Airflow workflow (DAG).

beginner

How do you define a Python function to use with PythonOperator?

You write a normal Python function that contains the logic you want to run as a task.

beginner

Which argument do you use to pass your Python function to PythonOperator?

You use the 'python_callable' argument to give PythonOperator your function.

intermediate

Can PythonOperator pass parameters to your Python function? If yes, how?

Yes, by using 'op_args' for positional arguments or 'op_kwargs' for keyword arguments.

intermediate

What happens if your Python function raises an error during execution in PythonOperator?

The task fails and Airflow marks it as failed, so you can see the error in the logs.

Which argument is used to specify the Python function in PythonOperator?

A callable_function

B task_function

C python_callable

D function_name

The 'python_callable' argument is the correct way to pass your Python function to PythonOperator.

How can you pass keyword arguments to the Python function in PythonOperator?

A Using 'op_kwargs' argument

B Using 'kwargs' argument

C Using 'args' argument

D Using 'params' argument

'op_kwargs' is used to pass keyword arguments to the Python function.

What is the result if the Python function raises an exception during task execution?

A Task retries automatically without failure

B Task is skipped

C Task is marked successful

D Task is marked failed

If an exception occurs, the task fails and Airflow marks it as failed.

Which Airflow component runs the PythonOperator task?

A Scheduler

B Executor

C Webserver

D Metastore

The Executor runs the tasks including PythonOperator tasks.

Can PythonOperator be used to run shell commands directly?

A No, use BashOperator for shell commands

B No, use Python scripts only

C Yes, but only with special flags

D Yes, directly

For shell commands, BashOperator is the right choice, not PythonOperator.

Explain how to create a PythonOperator task that runs a custom Python function with parameters.

Think about how you connect your function to the operator and pass data.

Describe what happens when a PythonOperator task fails due to an error in the Python function.

Consider Airflow's behavior on task failure.