Integration with PagerDuty and Slack in Apache Airflow - Mini Project: Build & Apply

Integration with PagerDuty and Slack

📖 Scenario: You are a DevOps engineer setting up alert notifications for your Airflow workflows. You want to send alerts to PagerDuty for incident management and to Slack for team communication.

🎯 Goal: Build a simple Airflow DAG that integrates with PagerDuty and Slack by defining the necessary configuration and tasks to send alerts.

📋 What You'll Learn

Create a dictionary with PagerDuty and Slack configuration details

Add a variable for the alert message content

Define Airflow tasks to send alerts to PagerDuty and Slack

Print the task IDs to confirm setup

💡 Why This Matters

🌍 Real World

In real DevOps environments, integrating Airflow with PagerDuty and Slack helps teams quickly respond to workflow failures and incidents.

💼 Career

Knowing how to configure alerting integrations is essential for DevOps engineers to maintain system reliability and effective communication.

1

Create Alert Configuration Dictionary

Create a dictionary called alert_config with these exact entries: 'pagerduty_integration_key': 'abc123' and 'slack_webhook_url': 'https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX'.

Apache Airflow

# Create the alert_config dictionary with PagerDuty and Slack keys
# Your code here

Hint

Use a Python dictionary with the exact keys and values given.

2

Add Alert Message Variable

Add a variable called alert_message and set it to the string 'Airflow task failed! Immediate attention required.'.

Apache Airflow

alert_config = {
    'pagerduty_integration_key': 'abc123',
    'slack_webhook_url': 'https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX'
}
# Define the alert_message string below
# Your code here

Hint

Use a simple string variable with the exact message.

3

Define Airflow Tasks for PagerDuty and Slack Alerts

Define two Airflow PythonOperator tasks: pagerduty_alert and slack_alert. Use python_callable functions named send_pagerduty_alert and send_slack_alert respectively. Assume these functions are already defined. Use dag=dag in both tasks.

Apache Airflow

alert_config = {
    'pagerduty_integration_key': 'abc123',
    'slack_webhook_url': 'https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX'
}
alert_message = 'Airflow task failed! Immediate attention required.'

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime

def send_pagerduty_alert():
    pass

def send_slack_alert():
    pass

dag = DAG('alert_integration', start_date=datetime(2024, 1, 1), schedule_interval='@daily')

# Define pagerduty_alert and slack_alert tasks below
# Your code here

Hint

Use PythonOperator with the exact task IDs and callable names.

4

Print Task IDs to Confirm Setup

Write two print statements to display the task IDs of pagerduty_alert and slack_alert.

Apache Airflow

alert_config = {
    'pagerduty_integration_key': 'abc123',
    'slack_webhook_url': 'https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX'
}
alert_message = 'Airflow task failed! Immediate attention required.'

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime

def send_pagerduty_alert():
    pass

def send_slack_alert():
    pass

dag = DAG('alert_integration', start_date=datetime(2024, 1, 1), schedule_interval='@daily')

pagerduty_alert = PythonOperator(
    task_id='pagerduty_alert',
    python_callable=send_pagerduty_alert,
    dag=dag
)

slack_alert = PythonOperator(
    task_id='slack_alert',
    python_callable=send_slack_alert,
    dag=dag
)

# Print the task IDs below
# Your code here

Hint

Use print to show the task_id of each task.