Managed Airflow (MWAA, Cloud Composer, Astronomer) - Mini Project: Build & Apply

Basic Workflow with Managed Airflow

📖 Scenario: You are working with a managed Airflow service like MWAA, Cloud Composer, or Astronomer. Your goal is to create a simple workflow (DAG) that runs daily and prints a greeting message.

🎯 Goal: Build a basic Airflow DAG with one task that prints 'Hello from Managed Airflow!'. This will help you understand how to define workflows in a managed Airflow environment.

📋 What You'll Learn

Create a DAG named hello_managed_airflow with a daily schedule.

Add a Python task named print_hello that prints the greeting message.

Use the PythonOperator to define the task.

Set the DAG's start date to January 1, 2024.

Print the task output in the final step.

💡 Why This Matters

🌍 Real World

Managed Airflow services like MWAA, Cloud Composer, and Astronomer help teams run workflows without managing Airflow infrastructure. Creating DAGs is the core way to automate tasks.

💼 Career

Knowing how to write and configure DAGs in managed Airflow is essential for DevOps engineers and data engineers to automate pipelines reliably and efficiently.

1

Create the DAG object

Import DAG from airflow and create a DAG object called hello_managed_airflow with schedule_interval='@daily' and start_date=datetime(2024, 1, 1). Import datetime from datetime.

Apache Airflow

from airflow import DAG
from datetime import datetime

# Create the DAG object
# Your code here

Hint

Use DAG('hello_managed_airflow', schedule_interval='@daily', start_date=datetime(2024, 1, 1)) to create the DAG.

2

Define the Python function to run

Create a Python function called print_hello_func that prints the exact text 'Hello from Managed Airflow!'.

Apache Airflow

from airflow import DAG
from datetime import datetime

hello_managed_airflow = DAG(
    'hello_managed_airflow',
    schedule_interval='@daily',
    start_date=datetime(2024, 1, 1)
)

# Define the Python function
# Your code here

Hint

Define a function with def print_hello_func(): and inside it use print('Hello from Managed Airflow!').

3

Create the PythonOperator task

Import PythonOperator from airflow.operators.python. Create a task called print_hello using PythonOperator with task_id='print_hello', python_callable=print_hello_func, and assign it to the hello_managed_airflow DAG.

Apache Airflow

from airflow import DAG
from datetime import datetime
from airflow.operators.python import PythonOperator

hello_managed_airflow = DAG(
    'hello_managed_airflow',
    schedule_interval='@daily',
    start_date=datetime(2024, 1, 1)
)

def print_hello_func():
    print('Hello from Managed Airflow!')

# Create the PythonOperator task
# Your code here

Hint

Use PythonOperator(task_id='print_hello', python_callable=print_hello_func, dag=hello_managed_airflow) to create the task.

4

Print the task output

Add a print statement to display the text 'DAG hello_managed_airflow with task print_hello is ready.'.

Apache Airflow

from airflow import DAG
from datetime import datetime
from airflow.operators.python import PythonOperator

hello_managed_airflow = DAG(
    'hello_managed_airflow',
    schedule_interval='@daily',
    start_date=datetime(2024, 1, 1)
)

def print_hello_func():
    print('Hello from Managed Airflow!')

print_hello = PythonOperator(
    task_id='print_hello',
    python_callable=print_hello_func,
    dag=hello_managed_airflow
)

# Print the confirmation message
# Your code here

Hint

Use print('DAG hello_managed_airflow with task print_hello is ready.') to show the message.