DAG concept (Directed Acyclic Graph) in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding DAG Structure in Airflow

Which statement best describes the structure of a DAG in Apache Airflow?

A A DAG is a collection of tasks with dependencies that form a cycle allowing repeated execution paths.

B A DAG is a linear sequence of tasks executed one after another without any branching.

C A DAG is a set of tasks connected in a way that no cycles exist, ensuring a clear start and end.

D A DAG is a random graph where tasks can depend on each other in any order.

💡 Hint

Think about what 'acyclic' means in the context of graph structures.

A DAG (Directed Acyclic Graph) means tasks are connected with directed edges and no cycles exist, so the workflow has a clear direction and no loops.

💻 Command Output

intermediate

Output of DAG Task Execution Order

Given the following Airflow DAG snippet, what is the order of task execution?

Apache Airflow

from airflow import DAG
from airflow.operators.dummy import DummyOperator
from datetime import datetime

dag = DAG('example_dag', start_date=datetime(2024, 1, 1))

t1 = DummyOperator(task_id='task1', dag=dag)
t2 = DummyOperator(task_id='task2', dag=dag)
t3 = DummyOperator(task_id='task3', dag=dag)

t1 >> [t2, t3]

A task1 runs first, then task2 and task3 run in parallel.

B task3 runs first, then task1 and task2 run in parallel.

C task2 runs first, then task1 and task3 run in parallel.

D task1, task2, and task3 run all at the same time.

💡 Hint

Look at the dependency operator '>>' and what it means for task order.

The '>>' operator means task1 must finish before task2 and task3 start. Tasks 2 and 3 can run at the same time after task1.

❓ Configuration

advanced

Identifying Invalid DAG Configuration

Which of the following DAG definitions will cause an error due to cycle detection in Airflow?

Apache Airflow

from airflow import DAG
from airflow.operators.dummy import DummyOperator
from datetime import datetime

dag = DAG('cycle_dag', start_date=datetime(2024, 1, 1))

t1 = DummyOperator(task_id='task1', dag=dag)
t2 = DummyOperator(task_id='task2', dag=dag)
t3 = DummyOperator(task_id='task3', dag=dag)

# Dependencies to check

A [t1, t2] >> t3

B t1 >> t2 >> t3

C t1 >> [t2, t3]

D t1 >> t2 >> t3 >> t1

💡 Hint

Check if any task depends on itself directly or indirectly.

Option D creates a cycle because task1 depends on task3, which depends back on task1, causing a loop. Airflow does not allow cycles in DAGs.

❓ Troubleshoot

advanced

Troubleshooting a DAG with Missing Task Dependencies

You have a DAG where task3 runs before task2, but your code specifies t1 >> t2 >> t3. What is the most likely cause?

A The DAG file is not saved, so Airflow is running an old version without dependencies.

B Airflow allows tasks to run in any order regardless of dependencies.

C The start_date is set in the future, so tasks run out of order.

D The tasks are defined without a DAG object.

💡 Hint

Think about how Airflow reads DAG files and updates task dependencies.

If the DAG file is not saved after adding dependencies, Airflow will run the last saved version which may not have the correct order, causing tasks to run out of sequence.

🔀 Workflow

expert

Determining the Number of Tasks Executed in a Complex DAG

Consider a DAG with tasks: t1, t2, t3, t4, t5. The dependencies are:

• t1 >> t2
• t1 >> t3
• t2 >> t4
• t3 >> t4
• t4 >> t5

If task t3 fails and depends_on_past=False, how many tasks will run successfully in the next DAG run?

A 3 tasks will run successfully: t1, t2, and t5.

B 2 tasks will run successfully: t1 and t2.

C 4 tasks will run successfully: t1, t2, t3, and t4.

D Only 1 task will run successfully: t1.

💡 Hint

Consider which tasks depend on the failed task and how failure affects downstream tasks.

Task t3 fails, so t4 cannot run because it depends on both t2 and t3. t5 depends on t4, so it also won't run. Only t1 and t2 run successfully.

Practice

1. What is a DAG in Apache Airflow?

easy

A. A tool to store data in a database

B. A programming language for writing tasks

C. A type of server used to run Airflow

D. A Directed Acyclic Graph that defines task order without loops

Solution

1. Step 1: Understand the meaning of DAG

   DAG stands for Directed Acyclic Graph, which means tasks are connected in one direction without cycles or loops.

2. Step 2: Relate DAG to Airflow

   In Airflow, DAGs organize tasks so they run in a specific order safely and predictably.

3. Final Answer:

   A Directed Acyclic Graph that defines task order without loops -> Option D

4. Quick Check:

   DAG = Directed Acyclic Graph [OK]

Hint: Remember DAG means tasks flow one way, no loops [OK]

Common Mistakes:

• Thinking DAG is a database or server
• Confusing DAG with a programming language
• Assuming tasks can loop back in DAG

2. Which of the following is the correct way to define a DAG in Airflow using Python?

easy

A. dag = DAG('my_dag', schedule_interval='@daily')

B. dag = DAG('my_dag', run_every='daily')

C. dag = DAG('my_dag', interval='daily')

D. dag = DAG('my_dag', schedule='daily')

Solution

1. Step 1: Recall Airflow DAG parameters

   The correct parameter to set the schedule is schedule_interval.

2. Step 2: Match the correct syntax

   Only dag = DAG('my_dag', schedule_interval='@daily') uses schedule_interval='@daily', which is valid syntax.

3. Final Answer:

   dag = DAG('my_dag', schedule_interval='@daily') -> Option A

4. Quick Check:

   Use schedule_interval to set DAG schedule [OK]

Hint: Use schedule_interval, not run_every or schedule [OK]

Common Mistakes:

• Using wrong parameter names like run_every
• Forgetting quotes around schedule value
• Using invalid schedule strings

3. Given this DAG snippet, what is the order of task execution?

task1 >> task2 >> task3

medium

A. task3 runs first, then task2, then task1

B. All tasks run at the same time

C. task1 runs first, then task2, then task3

D. task2 runs first, then task1, then task3

Solution

1. Step 1: Understand the '>>' operator in Airflow

   The '>>' operator means 'set downstream', so task1 runs before task2, and task2 before task3.

2. Step 2: Determine execution order

   Tasks run in the order task1, then task2, then task3.

3. Final Answer:

   task1 runs first, then task2, then task3 -> Option C

4. Quick Check:

   task1 >> task2 >> task3 means sequential order [OK]

Hint: Remember '>>' means run left task before right task [OK]

Common Mistakes:

• Thinking '>>' means run tasks in parallel
• Reversing the order of tasks
• Assuming tasks run simultaneously

4. What is wrong with this DAG task dependency code?

task1.set_downstream(task2)
task2.set_downstream(task1)

medium

A. It creates a cycle, which is not allowed in DAGs

B. The syntax for set_downstream is incorrect

C. Tasks must be named differently

D. No error, this is valid

Solution

1. Step 1: Analyze the dependencies

   task1 depends on task2, and task2 depends on task1, creating a loop.

2. Step 2: Understand DAG rules

   DAGs cannot have cycles; tasks must not depend on each other circularly.

3. Final Answer:

   It creates a cycle, which is not allowed in DAGs -> Option A

4. Quick Check:

   DAGs must be acyclic, no loops allowed [OK]

Hint: Check for circular dependencies to avoid cycles [OK]

Common Mistakes:

• Ignoring circular dependencies
• Assuming set_downstream syntax is wrong
• Thinking task names cause errors

5. You want to create a DAG where task3 runs only after both task1 and task2 complete. Which code correctly sets this dependency?

hard

A. task3 >> task1
task3 >> task2

B. task1 >> task3
task2 >> task3

C. task1 << task3
task2 << task3

D. task3.set_downstream([task1, task2])

Solution

1. Step 1: Understand dependency direction

   To run task3 after task1 and task2, task1 and task2 must be upstream of task3.

2. Step 2: Check correct syntax

   Using '>>' from task1 and task2 to task3 sets task3 to run after both complete.

3. Final Answer:

   task1 >> task3 task2 >> task3 -> Option B

4. Quick Check:

   Upstream tasks >> downstream task [OK]

Hint: Use 'task1 >> task3' and 'task2 >> task3' for multiple dependencies [OK]

Common Mistakes:

• Reversing dependency arrows
• Using set_downstream incorrectly with lists
• Assuming task3 runs before task1 and task2