Cloud Run jobs for batch work in GCP - Commands & Configuration

Introduction

Sometimes you need to run tasks that start, do some work, and then stop. Cloud Run jobs let you run these batch tasks easily without managing servers.

When you want to process a list of files one time and then stop.

When you need to run a script that cleans up old data once a day.

When you want to run a report that gathers information and then finishes.

When you want to run a task that sends emails to users in batches.

When you want to run a job that imports data from one place to another without keeping a server running.

Config File - job.yaml

job.yaml

apiVersion: run.googleapis.com/v1
kind: Job
metadata:
  name: example-batch-job
  namespace: default
spec:
  template:
    spec:
      containers:
      - image: gcr.io/cloudrun/hello
        command: ["/bin/sh", "-c", "echo Hello from batch job"]
      restartPolicy: Never
  parallelism: 1
  taskCount: 1

This YAML file defines a Cloud Run job named example-batch-job. It runs a container from the image gcr.io/cloudrun/hello that prints a message. The job runs one task at a time (parallelism: 1) and runs only one task total (taskCount: 1). The restartPolicy: Never means the task won't restart if it fails.

Commands

This command creates a Cloud Run job named example-batch-job using the specified container image and command. It sets the region to us-central1.

Terminal

gcloud run jobs create example-batch-job --image gcr.io/cloudrun/hello --command "/bin/sh" --args "-c","echo Hello from batch job" --region us-central1

Expected Output

Created job [example-batch-job] in region [us-central1].

→

--image - Specifies the container image to run.

→

--command - Overrides the container's default command.

→

--args - Arguments passed to the command.

This command starts running the example-batch-job job in the specified region.

Terminal

gcloud run jobs execute example-batch-job --region us-central1

Expected Output

Started execution [example-batch-job-00001] of job [example-batch-job].

→

--region - Specifies the region where the job runs.

This command shows details about the job execution, including status and logs.

Terminal

gcloud run jobs executions describe example-batch-job-00001 --region us-central1

Expected Output

name: projects/PROJECT_ID/locations/us-central1/jobs/example-batch-job/executions/example-batch-job-00001

status:

  conditions:

  - type: Completed

    status: "True"

  completionTime: '2024-06-01T12:00:00Z'

  startTime: '2024-06-01T11:59:50Z'

  succeededCount: 1

→

--region - Specifies the region of the job execution.

This command lists all Cloud Run jobs in the specified region to verify the job exists.

Terminal

gcloud run jobs list --region us-central1

Expected Output

NAME               REGION        LAST RUN

example-batch-job  us-central1   2024-06-01T12:00:00Z

→

--region - Specifies the region to list jobs from.

Key Concept

If you remember nothing else from this pattern, remember: Cloud Run jobs let you run one-time or batch tasks easily without managing servers.

Common Mistakes

Trying to run a Cloud Run job without creating it first.

The job must exist before you can execute it, or the command will fail.

Always create the job with 'gcloud run jobs create' before executing it.

Not specifying the region consistently in commands.

Cloud Run jobs are regional, so commands must use the same region or they won't find the job.

Use the --region flag with the same region for all job commands.

Setting restartPolicy to Always in a batch job.

Batch jobs should not restart tasks automatically; this can cause unexpected repeated runs.

Set restartPolicy to Never for batch jobs to run tasks only once.

Summary

Create a Cloud Run job with a container image and command using 'gcloud run jobs create'.

Run the job once with 'gcloud run jobs execute'.

Check the job execution status with 'gcloud run jobs executions describe'.

List all jobs in a region with 'gcloud run jobs list' to verify job presence.