Task groups for visual organization in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Purpose of Task Groups in Airflow

What is the main purpose of using Task Groups in an Airflow DAG?

A To replace the need for defining task dependencies between tasks

B To execute all tasks in the group simultaneously regardless of dependencies

C To visually organize related tasks into a single group for better readability in the Airflow UI

D To automatically retry failed tasks within the group without manual configuration

💡 Hint

Think about how Airflow UI shows tasks and how grouping helps.

Task Groups help organize related tasks visually in the Airflow UI, making complex DAGs easier to understand. They do not change execution behavior or dependencies.

💻 Command Output

intermediate

Output of Task Group Structure in Airflow UI

Given the following Airflow DAG snippet using Task Groups, what will be the visual structure shown in the Airflow UI?

Apache Airflow

from airflow import DAG
from airflow.operators.dummy import DummyOperator
from airflow.utils.task_group import TaskGroup
from datetime import datetime

default_args = {'start_date': datetime(2024, 1, 1)}

dag = DAG('example_dag', default_args=default_args, schedule_interval='@daily')

start = DummyOperator(task_id='start', dag=dag)
end = DummyOperator(task_id='end', dag=dag)

with TaskGroup('group1', dag=dag) as group1:
    t1 = DummyOperator(task_id='task1')
    t2 = DummyOperator(task_id='task2')

start >> group1 >> end

A The UI shows 'start' connected directly to 'task1' and 'task2' with no grouping

B The UI shows five separate tasks: 'start', 'group1', 'task1', 'task2', and 'end' all ungrouped

C The UI shows only 'start' and 'end' tasks because Task Groups hide tasks by default

D The UI shows three nodes: 'start', a collapsed group named 'group1' containing 'task1' and 'task2', and 'end' connected in sequence

💡 Hint

Task Groups create a collapsible group in the UI containing the tasks inside.

Task Groups appear as a single collapsible node in the Airflow UI. Inside, the grouped tasks are shown. The sequence is start → group1 → end.

❓ Configuration

advanced

Correct Task Group Definition Syntax

Which of the following code snippets correctly defines a Task Group named 'processing' with two dummy tasks inside in Airflow 2.5+?

A

processing = TaskGroup('processing')
with processing:
    task_a = DummyOperator(task_id='task_a')
    task_b = DummyOperator(task_id='task_b')

B

processing = TaskGroup('processing')
processing.task_a = DummyOperator(task_id='task_a')
processing.task_b = DummyOperator(task_id='task_b')

C

with TaskGroup('processing') as processing:
    task_a = DummyOperator(task_id='task_a')
    task_b = DummyOperator(task_id='task_b')

D

with TaskGroup('processing'):
    task_a = DummyOperator(task_id='task_a')
    task_b = DummyOperator(task_id='task_b')

💡 Hint

TaskGroup can be used as a context manager assigned to a variable.

The correct way is to create a TaskGroup instance, then use it as a context manager to define tasks inside. Option A does this correctly.

❓ Troubleshoot

advanced

Error When Using Task Groups Without DAG Context

What error will occur if you define a Task Group and tasks inside it without associating them with a DAG or DAG context?

A AirflowException: TaskGroup must be used within a DAG context

B TypeError: 'NoneType' object is not iterable

C AttributeError: 'TaskGroup' object has no attribute 'dag'

D ValueError: Task IDs must be unique within a DAG

💡 Hint

Task Groups require a DAG context to know where they belong.

Task Groups must be defined inside a DAG context or passed a DAG explicitly. Otherwise, Airflow raises an AirflowException indicating the missing DAG context.

🔀 Workflow

expert

Task Group Dependency Behavior

Consider two Task Groups, groupA and groupB, each with multiple tasks. If you set the dependency groupA >> groupB, what is the effect on the tasks inside these groups?

A Only the first task in groupA must complete before the first task in groupB starts

B All tasks in groupA must complete before any task in groupB starts

C Tasks in groupA and groupB run in parallel ignoring the dependency

D Only tasks with matching task_ids in both groups depend on each other

💡 Hint

Task Group dependencies apply to all tasks inside the groups.

Setting groupA >> groupB creates dependencies from every task in groupA to every task in groupB, so all groupA tasks must finish before groupB tasks start.