Unit testing DAGs in Apache Airflow - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the Airflow DAG class.

Apache Airflow

from airflow.models import [1]

Drag options to blanks, or click blank then click option'

ATaskInstance

BVariable

CBaseOperator

DDAG

Attempts:

3 left

💡 Hint

Common Mistakes

Importing TaskInstance instead of DAG

Using BaseOperator which is for tasks

The DAG class is imported from airflow.models to define workflows.

2fill in blank

medium

Complete the code to create a DAG object with id 'test_dag'.

Apache Airflow

dag = [1](dag_id='test_dag', start_date=datetime(2024, 1, 1))

Drag options to blanks, or click blank then click option'

ATaskInstance

BBaseOperator

CDAG

DVariable

Attempts:

3 left

💡 Hint

Common Mistakes

Using BaseOperator instead of DAG

Forgetting to set dag_id

The DAG object is created by calling DAG() with parameters like dag_id and start_date.

3fill in blank

hard

Fix the error in the test code to check if the DAG has the correct id.

Apache Airflow

assert dag.[1] == 'test_dag'

Drag options to blanks, or click blank then click option'

Adag_id

Bid

Cname

Ddag_name

Attempts:

3 left

💡 Hint

Common Mistakes

Using id or name instead of dag_id

Trying to access dag_name which is invalid

The DAG's unique identifier is accessed via the dag_id attribute.

4fill in blank

hard

Fill both blanks to create a task using PythonOperator with a simple callable.

Apache Airflow

task = PythonOperator(task_id='print_hello', python_callable=[1], dag=[2])

Drag options to blanks, or click blank then click option'

Aprint_hello

Bdag

Cprint

Dstart_date

Attempts:

3 left

💡 Hint

Common Mistakes

Using print instead of a function name

Passing start_date instead of dag

The python_callable parameter expects a function like print_hello, and dag is the DAG object.

5fill in blank

hard

Fill all three blanks to assert the task has the correct task_id and belongs to the DAG.

Apache Airflow

assert task.[1] == 'print_hello'
assert task.[2] == [3]

Drag options to blanks, or click blank then click option'

Atask_id

Bdag

Ddag_id

Attempts:

3 left

💡 Hint

Common Mistakes

Using dag_id instead of dag for task's DAG

Mixing up attribute names

Task's task_id attribute holds its id, and dag attribute links to the DAG object.

Practice

1. What is the main purpose of unit testing DAGs in Airflow?

easy

A. To catch errors early before running workflows

B. To deploy DAGs to production automatically

C. To monitor task execution times

D. To schedule DAG runs at specific times

Solution

1. Step 1: Understand unit testing purpose

   Unit testing is done to find bugs or errors early in code before actual execution.

2. Step 2: Apply this to Airflow DAGs

   Unit testing DAGs helps catch errors in DAG definitions before running workflows in Airflow.

3. Final Answer:

   To catch errors early before running workflows -> Option A

4. Quick Check:

   Unit testing = catch errors early [OK]

Hint: Unit tests find bugs before running DAGs [OK]

Common Mistakes:

• Confusing unit testing with deployment
• Thinking unit tests schedule DAGs
• Mixing monitoring with testing

2. Which Python code snippet correctly loads DAGs for unit testing using Airflow's DagBag?

easy

A. dagbag = DagBag.load('/path/to/dags')

B. dagbag = DagBag(dag_folder='/path/to/dags')

C. dagbag = DagBag(folder='/path/to/dags')

D. dagbag = DagBag.get('/path/to/dags')

Solution

1. Step 1: Recall DagBag constructor usage

   DagBag takes a keyword argument dag_folder to specify DAGs folder path.

2. Step 2: Check each option

   Only dagbag = DagBag(dag_folder='/path/to/dags') uses correct syntax. Others use invalid methods (A,D) or incorrect keyword argument like folder (B).

3. Final Answer:

   dagbag = DagBag(dag_folder='/path/to/dags') -> Option B

4. Quick Check:

   DagBag(dag_folder=...) correct syntax [OK]

Hint: Use dag_folder keyword to load DAGs in DagBag [OK]

Common Mistakes:

• Using incorrect keyword argument (e.g., folder)
• Using non-existent methods like load() or get()
• Omitting dag_folder keyword

3. Given this test code snippet, what will be the output if the DAG named 'example_dag' has 3 tasks?

dagbag = DagBag(dag_folder='dags')
dag = dagbag.get_dag('example_dag')
print(len(dag.tasks))

medium

A. None

B. 0

C. 3

D. Error: DAG not found

Solution

1. Step 1: Load DAG using DagBag

   DagBag loads DAGs from 'dags' folder; get_dag('example_dag') returns the DAG object if found.

2. Step 2: Count tasks in the DAG

   len(dag.tasks) returns the number of tasks in the DAG, which is 3 as given.

3. Final Answer:

   3 -> Option C

4. Quick Check:

   len(dag.tasks) = 3 [OK]

Hint: len(dag.tasks) gives task count [OK]

Common Mistakes:

• Assuming get_dag returns None if DAG exists
• Confusing tasks with task instances
• Expecting error if DAG is present

4. You wrote this test code but it raises an AttributeError: 'NoneType' object has no attribute 'tasks'. What is the likely cause?

dagbag = DagBag(dag_folder='dags')
dag = dagbag.get_dag('missing_dag')
print(len(dag.tasks))

medium

A. The DAG 'missing_dag' does not exist in the folder

B. DagBag cannot load DAGs from 'dags' folder

C. len() cannot be used on dag.tasks

D. get_dag() returns a list, not a DAG object

Solution

1. Step 1: Analyze get_dag return value

   get_dag returns None if the DAG name is not found in the loaded DAGs.

2. Step 2: Understand AttributeError cause

   Trying to access dag.tasks when dag is None causes AttributeError.

3. Final Answer:

   The DAG 'missing_dag' does not exist in the folder -> Option A

4. Quick Check:

   get_dag returns None if DAG missing [OK]

Hint: Check if get_dag returns None before accessing tasks [OK]

Common Mistakes:

• Assuming get_dag always returns a DAG
• Thinking len() is invalid on tasks list
• Believing DagBag can't load DAGs from folder

5. You want to write a unit test to check that your DAG named 'data_pipeline' has exactly 5 tasks and no import errors. Which code snippet correctly performs this test?

hard

A. dagbag = DagBag(dag_folder='dags')
assert dagbag.import_errors == {}
dag = dagbag.get_dag('data_pipeline')
assert len(dag.tasks) == 0

B. dagbag = DagBag('dags')
assert dagbag.import_errors == None
dag = dagbag.get_dag('data_pipeline')
assert len(dag.tasks) == 5

C. dagbag = DagBag(dag_folder='dags')
assert dagbag.import_errors != {}
dag = dagbag.get_dag('data_pipeline')
assert len(dag.tasks) == 5

D. dagbag = DagBag(dag_folder='dags')
assert dagbag.import_errors == {}
dag = dagbag.get_dag('data_pipeline')
assert len(dag.tasks) == 5

Solution

1. Step 1: Check import errors correctly

   dagbag.import_errors is a dict; empty dict {} means no import errors.

2. Step 2: Verify task count assertion

   len(dag.tasks) == 5 checks that the DAG has exactly 5 tasks.

3. Final Answer:

   dagbag = DagBag(dag_folder='dags') assert dagbag.import_errors == {} dag = dagbag.get_dag('data_pipeline') assert len(dag.tasks) == 5 -> Option D

4. Quick Check:

   import_errors == {} and task count 5 [OK]

Hint: Assert import_errors == {} and task count matches [OK]

Common Mistakes:

• Checking import_errors against None instead of {}
• Asserting wrong task count
• Confusing import_errors presence logic