Azure operators in Apache Airflow - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the Azure Blob Storage operator in Airflow.

Apache Airflow

from airflow.providers.microsoft.azure.operators.blob import [1]

Drag options to blanks, or click blank then click option'

AAzureBlobStorageOperator

BAzureBlobOperator

CAzureBlobStorageHook

DAzureBlobClient

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'AzureBlobOperator' which does not exist.

Confusing operator with hook or client classes.

The correct import is AzureBlobStorageOperator, which is the operator to interact with Azure Blob Storage in Airflow.

2fill in blank

medium

Complete the code to create an Azure Data Lake Storage operator with the correct task ID.

Apache Airflow

upload_task = AzureDataLakeStorageUploadOperator(task_id='[1]', ...)

Drag options to blanks, or click blank then click option'

Adelete_adls_file

Bdownload_from_adls

Cadls_list_files

Dupload_to_adls

Attempts:

3 left

💡 Hint

Common Mistakes

Using task IDs that describe downloading or deleting instead of uploading.

Using generic or unrelated task IDs.

The task ID should describe the task action. Since this operator uploads, upload_to_adls is the best choice.

3fill in blank

hard

Fix the error in the Azure Data Factory operator instantiation by completing the missing parameter.

Apache Airflow

adf_run = AzureDataFactoryRunPipelineOperator(pipeline_name='my_pipeline', [1]='my_resource_group', ...)

Drag options to blanks, or click blank then click option'

Aresource_group_name

BresourceGroup

Cresource_group

Dresourcegroupname

Attempts:

3 left

💡 Hint

Common Mistakes

Using camelCase or concatenated parameter names.

Misspelling the parameter name.

The correct parameter name is resource_group_name as per Airflow Azure Data Factory operator documentation.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that filters Azure Blob Storage blobs with size greater than 1000.

Apache Airflow

filtered_blobs = {blob.name: blob.size for blob in blobs if blob.size [1] [2]

Drag options to blanks, or click blank then click option'

A>

B1000

C<

D500

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<' instead of '>' causing wrong filtering.

Using wrong size value.

The condition filters blobs with size greater than 1000, so the operator is > and the value is 1000.

5fill in blank

hard

Fill all three blanks to create a dictionary comprehension that maps container names to blob counts where blob count is greater than 10.

Apache Airflow

container_blob_counts = {container.name: [1] for container, [2] in containers.items() if [3] > 10}

Drag options to blanks, or click blank then click option'

Ablob_count

Dcontainer_count

Attempts:

3 left

💡 Hint

Common Mistakes

Using different variable names causing errors.

Using unrelated variable names like container_count.

All blanks relate to the variable blob_count which holds the count of blobs per container. The comprehension filters containers with blob counts greater than 10.

Practice

1. What is the main purpose of Azure operators in Apache Airflow?

easy

A. To automate and manage Azure cloud services within Airflow workflows

B. To create user interfaces for Azure services

C. To replace Azure CLI commands manually

D. To monitor local server hardware performance

Solution

1. Step 1: Understand Azure operators role

   Azure operators are designed to automate tasks on Azure cloud services from Airflow.

2. Step 2: Compare options with this role

   Only To automate and manage Azure cloud services within Airflow workflows describes automation and management of Azure services, which matches the purpose.

3. Final Answer:

   To automate and manage Azure cloud services within Airflow workflows -> Option A

4. Quick Check:

   Azure operators automate Azure tasks = C [OK]

Hint: Azure operators automate Azure tasks in Airflow [OK]

Common Mistakes:

• Confusing operators with UI tools
• Thinking operators replace manual CLI commands only
• Assuming operators monitor local hardware

2. Which of the following is the correct way to import the Azure Data Factory operator in Airflow?

easy

A. import AzureDataFactoryRunPipelineOperator from airflow.microsoft.azure

B. from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator

C. from airflow.operators.azure.datafactory import RunPipelineOperator

D. from airflow.providers.azure.operators.datafactory import AzureRunPipelineOperator

Solution

1. Step 1: Recall correct import path for Azure Data Factory operator

   The official import path uses 'airflow.providers.microsoft.azure.operators.data_factory' and the class name 'AzureDataFactoryRunPipelineOperator'.

2. Step 2: Check each option for syntax and correctness

   from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator matches the correct module and class name exactly; others have wrong paths or class names.

3. Final Answer:

   from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator -> Option B

4. Quick Check:

   Correct import path and class name = A [OK]

Hint: Use full provider path with correct class name [OK]

Common Mistakes:

• Using incomplete or wrong import paths
• Incorrect class names or module names
• Mixing provider namespaces

3. Given this Airflow task using Azure VM operator:

run_vm = AzureVirtualMachineStartOperator(
    task_id='start_vm',
    resource_group_name='mygroup',
    vm_name='myvm'
)

What will happen when this task runs?

medium

A. It will create a new virtual machine named 'myvm'

B. It will stop the Azure virtual machine named 'myvm' in 'mygroup'

C. It will start the Azure virtual machine named 'myvm' in 'mygroup'

D. It will delete the virtual machine named 'myvm'

Solution

1. Step 1: Identify operator purpose

   The AzureVirtualMachineStartOperator is designed to start an existing VM.

2. Step 2: Analyze parameters and action

   The task specifies resource group and VM name, so it will start that VM, not stop, create, or delete it.

3. Final Answer:

   It will start the Azure virtual machine named 'myvm' in 'mygroup' -> Option C

4. Quick Check:

   Start operator starts VM = A [OK]

Hint: Operator name shows action: Start means start VM [OK]

Common Mistakes:

• Confusing start with stop or delete operators
• Assuming it creates a VM
• Ignoring resource group parameter

4. You wrote this Airflow task to delete an Azure storage container:

delete_container = AzureStorageDeleteOperator(
    task_id='delete_container',
    container_name='data',
    connection_string='my_conn_str'
)

But the task fails with an error about missing parameters. What is the likely cause?

medium

A. The operator does not support deleting containers, only blobs

B. The operator requires 'account_name' and 'account_key' instead of 'connection_string'

C. The operator requires 'resource_group_name' parameter

D. The operator requires the 'blob_name' parameter to delete a container

Solution

1. Step 1: Understand AzureStorageDeleteOperator capabilities

   This operator deletes blobs inside containers, not containers themselves.

2. Step 2: Analyze error cause

   Trying to delete a container causes failure because container deletion is unsupported by this operator.

3. Final Answer:

   The operator does not support deleting containers, only blobs -> Option A

4. Quick Check:

   Operator deletes blobs, not containers = D [OK]

Hint: Check operator docs for supported resource types [OK]

Common Mistakes:

• Assuming container deletion is supported
• Confusing connection parameters
• Adding unrelated parameters

5. You want to run an Azure Data Factory pipeline from Airflow and wait for its completion. Which operator and parameters combination is correct?

hard

A. AzureDataFactoryRunPipelineOperator with parameters: pipeline_id, resource_group, factory, async=True

B. AzureDataFactoryRunOperator with parameters: pipeline_name, resource_group_name, factory_name, wait_for_completion=True

C. AzureDataFactoryPipelineOperator with parameters: pipeline_name, resource_group_name, factory_name, wait=True

D. AzureDataFactoryRunPipelineOperator with parameters: pipeline_name, resource_group_name, factory_name, wait_for_termination=True

Solution

1. Step 1: Identify correct operator and parameters

   The official operator is AzureDataFactoryRunPipelineOperator. It uses 'pipeline_name', 'resource_group_name', 'factory_name', and 'wait_for_termination' to wait for completion.

2. Step 2: Compare options for exact names and flags

   AzureDataFactoryRunPipelineOperator with parameters: pipeline_name, resource_group_name, factory_name, wait_for_termination=True matches the correct operator and parameters exactly; others have wrong names or parameters.

3. Final Answer:

   AzureDataFactoryRunPipelineOperator with parameters: pipeline_name, resource_group_name, factory_name, wait_for_termination=True -> Option D

4. Quick Check:

   Correct operator and wait param = B [OK]

Hint: Use 'wait_for_termination=True' to wait for pipeline finish [OK]

Common Mistakes:

• Using wrong operator class names
• Incorrect parameter names
• Confusing async flags