Workflows for orchestration in GCP - Commands & Configuration

Introduction

When you have several tasks that need to run in order, it can be hard to keep track of them all. Workflows help by organizing these tasks so they run one after another automatically without mistakes.

When you want to automate a series of cloud tasks like calling APIs or moving data.

When you need to run tasks in a specific order without manual steps.

When you want to handle errors in a process and decide what to do next.

When you want to connect different cloud services smoothly.

When you want to save time by automating repeated sequences of actions.

Config File - workflow.yaml

workflow.yaml

main:
  params: [input]
  steps:
  - init:
      assign:
      - message: "Starting workflow"
  - call_api:
      call: http.get
      args:
        url: "https://jsonplaceholder.typicode.com/todos/1"
  - check_response:
      switch:
        - condition: ${call_api.status == 200}
          next: success
        - condition: ${true}
          next: failure
  - success:
      return: ${call_api.body}
  - failure:
      return: "API call failed"

This YAML file defines a simple workflow with these parts:

• main: The starting point with input parameters.
• steps: The list of actions to run in order.
• init: Sets a starting message.
• call_api: Calls an example API to get data.
• check_response: Checks if the API call was successful and decides next step.
• success/failure: Returns the API data or an error message.

Commands

This command uploads and deploys the workflow defined in workflow.yaml to Google Cloud in the us-central1 region.

Terminal

gcloud workflows deploy example-workflow --source=workflow.yaml --location=us-central1

Expected Output

Deploying workflow...done.

Workflow [example-workflow] deployed successfully.

→

--source - Specifies the workflow definition file to deploy.

→

--location - Sets the region where the workflow runs.

This command starts running the deployed workflow in the specified region.

Terminal

gcloud workflows run example-workflow --location=us-central1

Expected Output

Waiting for operation to finish...



+----------------------+----------------------+

|          FIELD       |        VALUE         |

+----------------------+----------------------+

| name                 | projects/project-id/locations/us-central1/workflows/example-workflow/executions/1234567890 |

| startTime            | 2024-06-01T12:00:00Z |

| state                | ACTIVE               |

+----------------------+----------------------+



Execution started.

→

--location - Specifies the region where the workflow is deployed.

This command checks the status and result of the workflow execution using its ID.

Terminal

gcloud workflows executions describe 1234567890 --workflow=example-workflow --location=us-central1

Expected Output

name: projects/project-id/locations/us-central1/workflows/example-workflow/executions/1234567890

startTime: '2024-06-01T12:00:00Z'

endTime: '2024-06-01T12:00:10Z'

state: SUCCEEDED

result: '{"userId":1,"id":1,"title":"delectus aut autem","completed":false}'

→

--workflow - Specifies which workflow's execution to describe.

→

--location - Specifies the region of the workflow.

Key Concept

If you remember nothing else from this pattern, remember: Workflows let you automate and control a series of cloud tasks step-by-step without manual effort.

Common Mistakes

Deploying the workflow without specifying the correct region using --location.

The workflow deploys to a default or wrong region, causing run commands to fail or not find the workflow.

Always use the same --location flag when deploying and running the workflow.

Not checking the execution status after starting the workflow.

You won't know if the workflow succeeded or failed and cannot see the output.

Use 'gcloud workflows executions describe' with the execution ID to check results.

Writing invalid YAML syntax in the workflow file.

The deployment fails because the workflow definition is not valid.

Use a YAML validator and follow the workflow syntax carefully before deploying.

Summary

Create a YAML file defining the workflow steps and logic.

Deploy the workflow to Google Cloud with gcloud workflows deploy.

Run the workflow using gcloud workflows run.

Check the workflow execution status and output with gcloud workflows executions describe.

Practice

1. What is the main purpose of Google Cloud Workflows?

easy

A. To store large amounts of data securely

B. To automate and coordinate multiple cloud services in a sequence

C. To create virtual machines for computing

D. To monitor network traffic in real-time

Solution

1. Step 1: Understand the role of Workflows

   Workflows are designed to automate tasks by connecting different cloud services step-by-step.

2. Step 2: Compare options with this role

   Options B, C, and D describe storage, computing, and monitoring, which are not the main purpose of Workflows.

3. Final Answer:

   To automate and coordinate multiple cloud services in a sequence -> Option B

4. Quick Check:

   Workflows automate cloud tasks = A [OK]

Hint: Workflows connect services step-by-step to automate tasks [OK]

Common Mistakes:

• Confusing Workflows with storage or compute services
• Thinking Workflows monitor network traffic
• Assuming Workflows create virtual machines

2. Which of the following is the correct way to start a step in a Google Cloud Workflow YAML?

easy

A. steps:

B. start:

C. main:

D. execute:

Solution

1. Step 1: Recall the YAML structure for Workflows

   The main entry point for a workflow is defined under the main: key.

2. Step 2: Check other options

   Options like steps:, start:, and execute: are not valid top-level keys for starting a workflow.

3. Final Answer:

   main: -> Option C

4. Quick Check:

   Workflow starts with main: in YAML [OK]

Hint: Workflows start with main: block in YAML [OK]

Common Mistakes:

• Using steps: instead of main:
• Confusing start: as the entry point
• Trying execute: which is invalid

3. Given this snippet of a workflow YAML:

main:
  steps:
  - call: http.get
    args:
      url: "https://example.com/api"
  - return: ${steps[0].body.status}

What will the workflow return if the HTTP GET response body is {"status": "success"}?

medium

A. "success"

B. {"status": "success"}

C. An error because of wrong syntax

D. null

Solution

1. Step 1: Understand the call and return steps

   The first step calls an HTTP GET and stores the response in steps[0]. The second step returns the status field from the response body.

2. Step 2: Extract the status value from the response

   The response body is {"status": "success"}, so steps[0].body.status is "success".

3. Final Answer:

   "success" -> Option A

4. Quick Check:

   Return status field from HTTP response = "success" [OK]

Hint: Return uses ${steps[index].body.field} to get JSON values [OK]

Common Mistakes:

• Returning the whole body instead of the status field
• Syntax errors in accessing steps array
• Assuming null if not parsed correctly

4. Identify the error in this workflow snippet:

main:
  steps:
  - call: http.get
    args:
      url: "https://example.com/api"
  - return: steps[0].body.status

medium

A. Missing quotes around the URL

B. Incorrect indentation of steps

C. http.get is not a valid call

D. Missing ${} around the expression in return

Solution

1. Step 1: Check expression syntax in return

   In Workflows YAML, expressions must be wrapped in ${} to be evaluated.

2. Step 2: Verify other parts

   Indentation is correct, http.get is valid, and URL is quoted properly.

3. Final Answer:

   Missing ${} around the expression in return -> Option D

4. Quick Check:

   Expressions need ${} to evaluate in return [OK]

Hint: Wrap expressions in ${} to evaluate in workflow YAML [OK]

Common Mistakes:

• Forgetting ${} around expressions
• Misaligning YAML indentation
• Assuming http.get is invalid
• Not quoting URLs

5. You want to create a workflow that calls two APIs in sequence, but only calls the second API if the first returns status "ok". Which workflow feature helps you implement this conditional logic?

hard

A. Using switch or if steps to check the first API's response

B. Using parallel steps to call both APIs at the same time

C. Using retry to repeat the first API call until status is ok

D. Using return to stop the workflow after the first call

Solution

1. Step 1: Understand conditional execution in workflows

   Workflows support switch or if steps to decide which steps run based on conditions.

2. Step 2: Match the requirement

   To call the second API only if the first returns "ok", use if or switch to check the first response before proceeding.

3. Final Answer:

   Using switch or if steps to check the first API's response -> Option A

4. Quick Check:

   Conditional steps use if/switch to control flow [OK]

Hint: Use if or switch to run steps conditionally in workflows [OK]

Common Mistakes:

• Using parallel steps which run both APIs simultaneously
• Using retry which repeats calls but doesn't conditionally run next
• Using return which ends workflow early