Manual triggers and parameters in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

Output of manual trigger with parameters in Airflow CLI

You manually trigger an Airflow DAG named data_pipeline with parameters using the CLI command:

airflow dags trigger data_pipeline --conf '{"run_date": "2024-06-01"}'

What will be the output message shown in the terminal?

A DAG data_pipeline triggered without parameters

B Error: Invalid JSON format in --conf parameter

C Created <DagRun data_pipeline @ 2024-06-01T00:00:00+00:00: manual__2024-06-01T00:00:00+00:00, externally triggered: True>

D No such DAG: data_pipeline

💡 Hint

Check the CLI output format when triggering DAGs with parameters.

When you trigger a DAG manually with the --conf parameter, Airflow creates a DagRun with the given configuration and shows a confirmation message including the DAG name and run time.

🧠 Conceptual

intermediate

Purpose of parameters in manual DAG triggers

Why do we use parameters when manually triggering an Airflow DAG?

A To change the DAG code without redeploying

B To automatically schedule the DAG for future runs

C To disable the DAG temporarily

D To customize the DAG run with specific input values for that execution

💡 Hint

Think about what changes between manual runs.

Parameters let you pass custom data to the DAG run, so tasks can behave differently based on those inputs.

🔀 Workflow

advanced

Correct order to manually trigger an Airflow DAG with parameters via UI

Arrange the steps in the correct order to manually trigger an Airflow DAG named etl_job with parameters using the Airflow web UI.

A 1,2,3,4

B 1,3,2,4

C 2,1,3,4

D 1,2,4,3

💡 Hint

Think about the natural navigation flow in the UI.

You first open the UI, select the DAG, then trigger it and provide parameters in the popup.

❓ Troubleshoot

advanced

Error when triggering DAG with invalid JSON parameters

You try to trigger a DAG manually with this CLI command:

airflow dags trigger my_dag --conf '{run_date: 2024-06-01}'

What error will you most likely see?

A KeyError: run_date not found

B JSONDecodeError: Expecting property name enclosed in double quotes

C TypeError: conf parameter must be a string

D No error, DAG triggered successfully

💡 Hint

Check the JSON format rules for keys and strings.

The JSON keys must be enclosed in double quotes. Missing quotes cause JSONDecodeError.

✅ Best Practice

expert

Best practice for passing sensitive parameters in manual DAG triggers

When manually triggering an Airflow DAG that requires sensitive parameters (like passwords), what is the best practice?

A Use Airflow Variables or Connections to store sensitive data and reference them in the DAG code

B Pass sensitive data directly in the --conf JSON parameter when triggering manually

C Write sensitive data in plain text inside the DAG Python file

D Send sensitive data via email to the Airflow admin before triggering

💡 Hint

Think about security and avoiding exposure of secrets.

Storing secrets in Airflow Variables or Connections keeps them secure and avoids exposing them in logs or CLI commands.