Testing custom operators in Apache Airflow - Time & Space Complexity

Time Complexity: Testing custom operators

O(n)

Understanding Time Complexity

When testing custom operators in Airflow, it's important to understand how the test execution time grows as the number of test cases or operator complexity increases.

We want to know how the time to run tests changes when we add more tests or more complex logic.

Scenario Under Consideration

Analyze the time complexity of the following test code snippet.

from airflow.models import BaseOperator
from airflow.utils.decorators import apply_defaults

class MyCustomOperator(BaseOperator):
    @apply_defaults
    def __init__(self, param, **kwargs):
        super().__init__(**kwargs)
        self.param = param

    def execute(self, context):
        return self.param * 2


def test_my_custom_operator():
    operator = MyCustomOperator(task_id='test', param=5)
    result = operator.execute(context={})
    assert result == 10

This code defines a simple custom operator and a test function that runs the operator's execute method once and checks the result.

Identify Repeating Operations

Look for loops or repeated calls in the test.

• Primary operation: Single call to execute method of the operator.
• How many times: Exactly once per test run.

How Execution Grows With Input

Since the test runs the operator once, the time grows directly with the number of tests.

Input Size (number of tests)	Approx. Operations
10	10 calls to execute
100	100 calls to execute
1000	1000 calls to execute

Pattern observation: The total test time grows linearly as you add more tests.

Final Time Complexity

Time Complexity: O(n)

This means the test time increases in direct proportion to the number of tests you run.

Common Mistake

[X] Wrong: "Running more tests won't affect total test time much because each test is small."

[OK] Correct: Even small tests add up, so doubling tests roughly doubles total time.

Interview Connect

Understanding how test time grows helps you write efficient tests and manage test suites well in real projects.

Self-Check

"What if we added a loop inside the execute method that runs proportional to an input size? How would the time complexity change?"