Creating custom operators in Apache Airflow - Quick Revision & Summary

Recall & Review

beginner

What is a custom operator in Apache Airflow?

A custom operator is a user-defined class that extends Airflow's BaseOperator to perform specific tasks not covered by built-in operators.

beginner

Which method must be implemented when creating a custom operator?

The execute() method, which contains the code that runs when the operator is triggered.

intermediate

How do you pass parameters to a custom operator?

Parameters are passed via the operator's constructor and stored as instance variables to be used in the execute() method.

intermediate

Why should you call super().__init__() in a custom operator's constructor?

To properly initialize the BaseOperator part of the class, ensuring Airflow can manage the operator's lifecycle and features.

beginner

What is the benefit of creating custom operators in Airflow?

They allow you to reuse complex or specific task logic easily across multiple DAGs, improving code organization and maintainability.

What base class should you extend to create a custom operator in Airflow?

A BaseOperator

B DAG

C TaskInstance

D PythonOperator

Custom operators extend BaseOperator to inherit core operator functionality.

Which method contains the code that runs when a custom operator executes?

A run()

B execute()

C start()

D trigger()

The execute() method is called by Airflow to run the operator's task.

Why is calling super().__init__() important in a custom operator?

A To run the task

B To initialize the DAG

C To start the scheduler

D To initialize the BaseOperator

It ensures the BaseOperator is properly initialized with Airflow's internal setup.

How do you pass custom parameters to your operator?

A Via the operator's constructor

B By setting environment variables

C Through the execute() method

D Using Airflow variables only

Parameters are passed when creating the operator instance via its constructor.

What is a main advantage of using custom operators?

A They reduce the number of DAGs

B They replace the need for sensors

C They allow reuse of task logic across DAGs

D They speed up the scheduler

Custom operators help reuse complex task logic, making DAGs cleaner and easier to maintain.

Explain the steps to create a custom operator in Airflow.

Think about class inheritance and method overriding.

Describe why and when you would create a custom operator instead of using built-in ones.

Consider task uniqueness and code reuse.