Azure operators in Apache Airflow - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

Output of Azure Data Factory Pipeline Run Operator

What is the output of the following Airflow task using AzureDataFactoryRunPipelineOperator when the pipeline run is successful?

Apache Airflow

run_pipeline = AzureDataFactoryRunPipelineOperator(
    task_id='run_pipeline',
    pipeline_name='example_pipeline',
    azure_data_factory_conn_id='azure_data_factory_default',
    resource_group_name='my_resource_group',
    factory_name='my_factory'
)

result = run_pipeline.execute({})
print(result)

A Returns a dictionary containing the pipeline run ID and status 'Succeeded'

B Returns None because execute method does not return anything

C Raises AirflowException due to missing pipeline parameters

D Returns a string with the pipeline name only

💡 Hint

Think about what the operator returns after triggering a pipeline run.

The AzureDataFactoryRunPipelineOperator triggers a pipeline and returns a dictionary with details including the run ID and status. This helps track the pipeline execution.

❓ Troubleshoot

intermediate

Troubleshooting Azure Blob Storage Hook Connection Error

You have an Airflow task using AzureBlobStorageHook but it fails with a connection error. Which of the following is the most likely cause?

Apache Airflow

hook = AzureBlobStorageHook(azure_blob_conn_id='azure_blob_default')
container_client = hook.get_container_client('mycontainer')
container_client.list_blobs()

A The hook does not support listing blobs in a container

B The container name 'mycontainer' does not exist in Azure Blob Storage

C The Azure connection ID 'azure_blob_default' is not configured or has wrong credentials

D The Airflow worker does not have internet access

💡 Hint

Check the connection configuration in Airflow UI.

A connection error usually means the credentials or connection ID is missing or incorrect in Airflow's connection settings.

❓ Configuration

advanced

Configuring Azure Data Lake Storage Gen2 Hook with OAuth

Which configuration snippet correctly sets up AzureDataLakeStorageHook to authenticate using OAuth with a service principal?

A

hook = AzureDataLakeStorageHook(
    account_name='storage-account',
    sas_token='sas-token'
)

B

hook = AzureDataLakeStorageHook(
    account_name='storage-account',
    auth_type='SharedKey',
    client_id='client-id'
)

C

hook = AzureDataLakeStorageHook(
    connection_string='DefaultEndpointsProtocol=https;AccountName=storage-account;AccountKey=key;EndpointSuffix=core.windows.net'
)

D

hook = AzureDataLakeStorageHook(
    tenant_id='tenant-id',
    client_id='client-id',
    client_secret='client-secret',
    account_name='storage-account',
    auth_type='OAuth'
)

💡 Hint

OAuth requires tenant ID, client ID, and client secret.

OAuth authentication for Azure Data Lake Storage Gen2 requires tenant ID, client ID, client secret, and specifying auth_type as 'OAuth'.

🔀 Workflow

advanced

Order of Tasks for Uploading File to Azure Blob Storage and Triggering Data Factory Pipeline

Given these Airflow tasks, which option shows the correct order to upload a file to Azure Blob Storage and then trigger an Azure Data Factory pipeline that processes the file?

A 3,1,2,4

B 3,2,1,4

C 1,3,2,4

D 2,1,3,4

💡 Hint

The file must be uploaded before the pipeline runs.

The workflow should start, upload the file, then trigger the pipeline, and finally end.

✅ Best Practice

expert

Best Practice for Handling Secrets in Azure Operators within Airflow

Which option describes the best practice for managing Azure credentials securely when using Azure operators in Airflow?

A Hardcode client secrets directly in the DAG Python files for easy access

B Store credentials in Airflow Connections and use environment variables or Azure Key Vault integration to avoid hardcoding secrets

C Use plain text files on the Airflow server to store Azure credentials and read them in the DAG

D Embed credentials in the operator parameters directly in the DAG code

💡 Hint

Think about security and secret management best practices.

Storing secrets in Airflow Connections and integrating with Azure Key Vault or environment variables keeps credentials secure and avoids exposure in code.