Concurrency and scaling in GCP - Commands & Configuration

Introduction

When many users try to use your app at the same time, it can slow down or crash. Concurrency and scaling help your app handle many users smoothly by running multiple tasks at once and adding more resources automatically.

When your website gets more visitors than usual and you want it to stay fast.

When your app needs to process many requests at the same time without waiting.

When you want your service to add more servers automatically during busy times.

When you want to save money by using fewer resources when traffic is low.

When you want to avoid crashes caused by too many users accessing your app.

Config File - app.yaml

app.yaml

runtime: python39
instance_class: F2
automatic_scaling:
  target_cpu_utilization: 0.65
  min_instances: 1
  max_instances: 5
  max_concurrent_requests: 50

This file configures an App Engine service on Google Cloud.

runtime sets the Python version.

instance_class chooses the server size.

automatic_scaling controls how the app adds or removes instances based on CPU use and request load.

max_concurrent_requests limits how many requests one instance handles at once to balance speed and resource use.

Commands

This command uploads and applies the configuration to Google App Engine, starting your app with the scaling settings.

Terminal

gcloud app deploy app.yaml

Expected Output

Services to deploy:

  descriptor: app.yaml



Beginning deployment...

Updating service [default]...

Waiting for operation to complete...



Deployed service [default] to [https://PROJECT_ID.uc.r.appspot.com]



You can stream logs from the command line by running:

  gcloud app logs tail -s default

This command shows the current running instances of your app, so you can see how many servers are active.

Terminal

gcloud app instances list

Expected Output

SERVICE  VERSION  INSTANCE  VM_ID                                   VM_IP           STATE

default  20240601  instance-1  1234567890abcdef1234567890abcdef  35.233.123.45   RUNNING

This command streams live logs from your app to watch how it handles requests and scales in real time.

Terminal

gcloud app logs tail -s default

Expected Output

2024-06-01 12:00:00 default[20240601]: Started request GET / from 203.0.113.1

2024-06-01 12:00:01 default[20240601]: Completed request GET / with status 200

→

-s - Specifies the service to get logs from

Key Concept

If you remember nothing else from this pattern, remember: scaling automatically adds or removes app instances based on demand to keep your app fast and stable.

Common Mistakes

Setting max_instances too low

Your app cannot add enough servers during high traffic, causing slow responses or errors.

Set max_instances high enough to handle peak traffic safely.

Not limiting max_concurrent_requests

One instance tries to handle too many requests at once, leading to slow processing or crashes.

Set max_concurrent_requests to a reasonable number to balance load per instance.

Forgetting to deploy after changing app.yaml

Your scaling settings do not update, so the app does not scale as expected.

Always run 'gcloud app deploy app.yaml' after editing the config file.

Summary

Create an app.yaml file to set automatic scaling rules for your app.

Deploy the app with 'gcloud app deploy app.yaml' to apply scaling settings.

Use 'gcloud app instances list' to check how many app instances are running.

Stream logs with 'gcloud app logs tail -s default' to monitor app behavior and scaling.

Practice

1. What does concurrency control in a Google Cloud service primarily manage?

easy

A. The network bandwidth available to the service

B. The total number of instances running

C. The number of requests one instance can handle at the same time

D. The amount of storage used by the service

Solution

1. Step 1: Understand concurrency meaning

   Concurrency means how many tasks or requests can happen at once in one place.

2. Step 2: Apply concurrency to cloud instances

   In cloud services, concurrency controls how many requests a single instance handles simultaneously.

3. Final Answer:

   The number of requests one instance can handle at the same time -> Option C

4. Quick Check:

   Concurrency = requests per instance [OK]

Hint: Concurrency = requests per instance at once [OK]

Common Mistakes:

• Confusing concurrency with total instances
• Thinking concurrency controls storage or bandwidth
• Mixing concurrency with scaling

2. Which of the following is the correct way to set maximum concurrency to 10 in a Google Cloud Run service using gcloud CLI?

easy

A. gcloud run services create SERVICE --max-concurrency=10

B. gcloud run services update SERVICE --concurrency=10

C. gcloud run services update SERVICE --max-instances=10

D. gcloud run services update SERVICE --concurrency-limit=10

Solution

1. Step 1: Identify correct flag for concurrency

   The correct flag to set concurrency in Cloud Run is --concurrency.

2. Step 2: Use update command for existing service

   To change concurrency, use 'gcloud run services update SERVICE --concurrency=10'.

3. Final Answer:

   gcloud run services update SERVICE --concurrency=10 -> Option B

4. Quick Check:

   Concurrency flag is --concurrency [OK]

Hint: Use --concurrency flag with update command [OK]

Common Mistakes:

• Using --max-concurrency which is invalid
• Confusing concurrency with max-instances
• Trying to set concurrency on service creation instead of update

3. Consider a Cloud Run service with max concurrency set to 5 and max instances set to 3. If 20 requests arrive simultaneously, how many instances will be used to handle the load?

medium

A. 5 instances

B. 4 instances

C. 20 instances

D. 3 instances

Solution

1. Step 1: Calculate needed instances for 20 requests with concurrency 5

   Each instance can handle 5 requests at once, so 20 requests need 20/5 = 4 instances.

2. Step 2: Apply max instances limit

   The max instances is set to 3, so the service cannot create more than 3 instances.

3. Final Answer:

   3 instances -> Option D

4. Quick Check:

   Max instances limit caps instance count [OK]

Hint: Max instances limit caps scaling even if more needed [OK]

Common Mistakes:

• Ignoring max instances limit and choosing 4
• Confusing concurrency with max instances
• Assuming one instance per request

4. You set concurrency to 80 on a Cloud Run service, but your app crashes under load. What is the most likely cause?

medium

A. The app cannot handle that many requests at once causing resource exhaustion

B. The max instances setting is too low

C. The service is not deployed in the right region

D. The concurrency setting must be at least 100

Solution

1. Step 1: Understand concurrency impact on app

   High concurrency means many requests handled simultaneously by one instance, increasing resource use.

2. Step 2: Link app crash to resource limits

   If the app crashes, it likely cannot handle that many requests at once, causing memory or CPU overload.

3. Final Answer:

   The app cannot handle that many requests at once causing resource exhaustion -> Option A

4. Quick Check:

   Too high concurrency can crash app [OK]

Hint: High concurrency can overload app resources [OK]

Common Mistakes:

• Blaming region instead of concurrency
• Assuming concurrency must be at least 100
• Ignoring app resource limits

5. You want your Cloud Run service to handle sudden traffic spikes efficiently without wasting resources during low traffic. Which combination of settings is best?

hard

A. Set concurrency to a moderate number (e.g., 10) and allow max instances to scale high

B. Set concurrency to 1 and max instances to 1

C. Set concurrency very high (e.g., 100) and max instances low (e.g., 2)

D. Disable concurrency and fix max instances to 10

Solution

1. Step 1: Balance concurrency and scaling for efficiency

   Moderate concurrency lets each instance handle multiple requests, saving resources.

2. Step 2: Allow max instances to scale high for spikes

   High max instances lets the service add instances when traffic spikes, keeping performance.

3. Final Answer:

   Set concurrency to a moderate number (e.g., 10) and allow max instances to scale high -> Option A

4. Quick Check:

   Moderate concurrency + high max instances = efficient scaling [OK]

Hint: Moderate concurrency + high max instances for spikes [OK]

Common Mistakes:

• Setting concurrency too low wastes instances
• Setting concurrency too high risks crashes
• Fixing max instances limits scaling