Cloud Run jobs for batch work in GCP - Mini Project: Build & Apply

Cloud Run Jobs for Batch Work

📖 Scenario: You work at a company that needs to process large batches of data files regularly. Instead of running these tasks on a server all the time, you want to use Google Cloud Run jobs to run these batch tasks only when needed. This saves money and makes your system more efficient.

🎯 Goal: Build a Cloud Run job configuration that defines a batch job to process data files. You will create the job definition, set a retry policy, configure the job to run with a specific container image, and finally deploy the job with the correct execution command.

📋 What You'll Learn

Create a Cloud Run job configuration dictionary with the exact job name and container image

Add a retry policy configuration to the job

Define the execution command for the job container

Complete the job deployment configuration with the correct Cloud Run job resource structure

💡 Why This Matters

🌍 Real World

Cloud Run jobs let companies run batch tasks on demand without keeping servers running all the time, saving money and simplifying operations.

💼 Career

Knowing how to configure and deploy Cloud Run jobs is useful for cloud engineers and developers who manage scalable batch processing workloads.

1

Create the initial Cloud Run job configuration

Create a dictionary called cloud_run_job with these exact entries: "name": "batch-data-processor" and "container_image": "gcr.io/my-project/data-processor:latest".

GCP

# Your code here

Need a hint?

Think of this dictionary as the basic info about your batch job: its name and which container image it uses.

2

Add a retry policy to the job configuration

Add a key "retry_policy" to the cloud_run_job dictionary with the value "retryOnFailure".

GCP

cloud_run_job = {
    "name": "batch-data-processor",
    "container_image": "gcr.io/my-project/data-processor:latest"
}
# Add retry_policy key below

Need a hint?

This setting tells Cloud Run to retry the job if it fails.

3

Define the execution command for the job container

Add a key "command" to the cloud_run_job dictionary with the value being a list of strings: ["python", "process_data.py"].

GCP

cloud_run_job = {
    "name": "batch-data-processor",
    "container_image": "gcr.io/my-project/data-processor:latest",
    "retry_policy": "retryOnFailure"
}
# Add command key below

Need a hint?

This command tells the container what to run when the job starts.

4

Complete the Cloud Run job deployment configuration

Create a dictionary called cloud_run_job_deployment with a key "job" whose value is the cloud_run_job dictionary.

GCP

cloud_run_job = {
    "name": "batch-data-processor",
    "container_image": "gcr.io/my-project/data-processor:latest",
    "retry_policy": "retryOnFailure",
    "command": ["python", "process_data.py"]
}
# Create cloud_run_job_deployment dictionary below

Need a hint?

This final dictionary wraps your job configuration for deployment to Cloud Run.