Testing custom operators in Apache Airflow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main purpose of testing custom operators in Airflow?

To ensure that the custom operator behaves as expected and integrates correctly within Airflow workflows before deploying to production.

beginner

Which Python module is commonly used to write unit tests for Airflow custom operators?

The unittest module is commonly used for writing unit tests in Python, including tests for Airflow custom operators.

intermediate

Why should you mock Airflow's context when testing custom operators?

Mocking Airflow's context allows you to simulate the runtime environment and test the operator's logic without running a full Airflow scheduler or executor.

intermediate

What is a simple way to test the execute() method of a custom operator?

Create a test case that calls the execute() method with a mocked context and asserts expected outcomes or side effects.

advanced

How can you verify that your custom operator handles errors correctly during testing?

By writing tests that simulate error conditions and asserting that the operator raises the expected exceptions or handles errors gracefully.

What is the first step when testing a custom Airflow operator?

A Mock the Airflow context

B Run the full Airflow scheduler

C Deploy the operator to production

D Write a DAG using the operator

Mocking the Airflow context allows isolated testing of the operator without needing the full Airflow environment.

Which method in a custom operator is usually tested to check its main functionality?

A execute()

B run()

C start()

D __init__()

The execute() method contains the core logic of the operator and is the main focus of testing.

Why is it important to mock external services in custom operator tests?

A To make tests fail intentionally

B To speed up tests and avoid dependency on external systems

C To deploy faster

D To avoid writing tests

Mocking external services ensures tests run quickly and reliably without depending on outside systems.

Which Python library is commonly used for mocking in tests?

A requests

B pandas

C flask

D mock

The mock library (built-in as unittest.mock) is used to replace parts of the system under test with mock objects.

What should a test assert after calling the execute() method of a custom operator?

A That the operator returns None

B That the operator deletes DAG files

C That the operator performs expected actions or returns expected results

D That the operator runs forever

Tests should verify that the operator performs its intended actions or returns expected results.

Describe the steps to write a unit test for a custom Airflow operator's execute() method.

Think about isolating the operator logic and simulating Airflow's runtime environment.

Explain why mocking is important when testing custom operators in Airflow.

Consider how tests can run without external dependencies.