Concurrency and scaling in GCP - Step-by-Step Execution

Process Flow - Concurrency and scaling

User Requests Arrive

↓

Load Balancer Distributes

↓

Multiple Instances Handle Requests

↓

Monitor Performance & Load

↓

Scale Up or Down Instances

↓

Maintain Service Availability

User requests come in and are spread by a load balancer to many instances. The system watches load and adds or removes instances to keep service smooth.

Execution Sample

GCP

1. User sends 10 requests
2. Load balancer sends 5 requests per instance
3. Instances process requests concurrently
4. Monitor detects high load
5. Add 2 more instances
6. Load balancer redistributes requests

This simulates how requests are handled concurrently by multiple instances and how scaling adds instances to handle more load.

Process Table

Step	Action	Instances Active	Requests per Instance	Total Requests	Scaling Event
1	10 user requests arrive	2	0	10	No
2	Load balancer distributes requests	2	5	10	No
3	Instances process requests concurrently	2	5	10	No
4	Monitor detects high CPU load	2	5	10	Yes - scale up
5	Add 2 more instances	4	0	10	Yes - added 2 instances
6	Load balancer redistributes requests	4	2.5	10	No
7	Instances process requests concurrently	4	2.5	10	No
8	Load decreases, monitor detects low load	4	2.5	10	Yes - scale down
9	Remove 2 instances	2	0	10	Yes - removed 2 instances
10	Load balancer redistributes requests	2	5	10	No
11	Instances process requests concurrently	2	5	10	No
12	No more scaling needed	2	5	10	No

💡 Load stabilizes, no further scaling needed

Status Tracker

Variable	Start	After Step 2	After Step 5	After Step 6	After Step 9	Final
Instances Active	2	2	4	4	2	2
Requests per Instance	0	5	0	2.5	0	5
Total Requests	10	10	10	10	10	10
Scaling Event	No	No	Yes - added 2	No	Yes - removed 2	No

Key Moments - 3 Insights

Why does the number of requests per instance drop after scaling up?

Because the total requests are shared among more instances, so each instance handles fewer requests (see execution_table step 6).

Why do we remove instances when load decreases?

To save resources and cost by having fewer instances when demand is low (see execution_table step 9).

Does scaling happen instantly when load changes?

No, monitoring detects load changes and triggers scaling events, which take some time to add or remove instances (see steps 4-5 and 8-9).

Visual Quiz - 3 Questions

Look at the execution_table at step 6, how many requests does each instance handle?

A 2.5

B 5

C 10

D 0

💡 Hint

Check the 'Requests per Instance' column at step 6 in the execution_table.

At which step does the system add more instances?

A Step 9

B Step 4

C Step 5

D Step 2

💡 Hint

Look for the 'Scaling Event' column indicating 'added 2 instances' in the execution_table.

If total requests increased to 20 at step 1, what would happen to requests per instance before scaling?

A It would be 20 per instance

B It would be 10 per instance

C It would be 5 per instance

D It would be 0

💡 Hint

Divide total requests by instances active before scaling (see variable_tracker and execution_table step 2).

Concept Snapshot

Concurrency and scaling in cloud means handling many user requests at once by spreading them across multiple instances.
A load balancer sends requests to instances.
Monitoring watches load and adds or removes instances to keep service smooth.
Scaling up means adding instances; scaling down means removing them.
This keeps the service fast and cost-effective.

Full Transcript

Concurrency and scaling in cloud infrastructure means that many user requests come in at the same time and are handled by multiple server instances working together. A load balancer spreads the requests evenly so no single instance is overloaded. The system monitors how busy the instances are. If they get too busy, it adds more instances to share the work. If they are not busy, it removes some instances to save resources. This process keeps the service running smoothly and efficiently. The execution table shows how requests are divided among instances and how scaling events add or remove instances based on load.

Practice

1. What does concurrency control in a Google Cloud service primarily manage?

easy

A. The network bandwidth available to the service

B. The total number of instances running

C. The number of requests one instance can handle at the same time

D. The amount of storage used by the service

Solution

1. Step 1: Understand concurrency meaning

   Concurrency means how many tasks or requests can happen at once in one place.

2. Step 2: Apply concurrency to cloud instances

   In cloud services, concurrency controls how many requests a single instance handles simultaneously.

3. Final Answer:

   The number of requests one instance can handle at the same time -> Option C

4. Quick Check:

   Concurrency = requests per instance [OK]

Hint: Concurrency = requests per instance at once [OK]

Common Mistakes:

• Confusing concurrency with total instances
• Thinking concurrency controls storage or bandwidth
• Mixing concurrency with scaling

2. Which of the following is the correct way to set maximum concurrency to 10 in a Google Cloud Run service using gcloud CLI?

easy

A. gcloud run services create SERVICE --max-concurrency=10

B. gcloud run services update SERVICE --concurrency=10

C. gcloud run services update SERVICE --max-instances=10

D. gcloud run services update SERVICE --concurrency-limit=10

Solution

1. Step 1: Identify correct flag for concurrency

   The correct flag to set concurrency in Cloud Run is --concurrency.

2. Step 2: Use update command for existing service

   To change concurrency, use 'gcloud run services update SERVICE --concurrency=10'.

3. Final Answer:

   gcloud run services update SERVICE --concurrency=10 -> Option B

4. Quick Check:

   Concurrency flag is --concurrency [OK]

Hint: Use --concurrency flag with update command [OK]

Common Mistakes:

• Using --max-concurrency which is invalid
• Confusing concurrency with max-instances
• Trying to set concurrency on service creation instead of update

3. Consider a Cloud Run service with max concurrency set to 5 and max instances set to 3. If 20 requests arrive simultaneously, how many instances will be used to handle the load?

medium

A. 5 instances

B. 4 instances

C. 20 instances

D. 3 instances

Solution

1. Step 1: Calculate needed instances for 20 requests with concurrency 5

   Each instance can handle 5 requests at once, so 20 requests need 20/5 = 4 instances.

2. Step 2: Apply max instances limit

   The max instances is set to 3, so the service cannot create more than 3 instances.

3. Final Answer:

   3 instances -> Option D

4. Quick Check:

   Max instances limit caps instance count [OK]

Hint: Max instances limit caps scaling even if more needed [OK]

Common Mistakes:

• Ignoring max instances limit and choosing 4
• Confusing concurrency with max instances
• Assuming one instance per request

4. You set concurrency to 80 on a Cloud Run service, but your app crashes under load. What is the most likely cause?

medium

A. The app cannot handle that many requests at once causing resource exhaustion

B. The max instances setting is too low

C. The service is not deployed in the right region

D. The concurrency setting must be at least 100

Solution

1. Step 1: Understand concurrency impact on app

   High concurrency means many requests handled simultaneously by one instance, increasing resource use.

2. Step 2: Link app crash to resource limits

   If the app crashes, it likely cannot handle that many requests at once, causing memory or CPU overload.

3. Final Answer:

   The app cannot handle that many requests at once causing resource exhaustion -> Option A

4. Quick Check:

   Too high concurrency can crash app [OK]

Hint: High concurrency can overload app resources [OK]

Common Mistakes:

• Blaming region instead of concurrency
• Assuming concurrency must be at least 100
• Ignoring app resource limits

5. You want your Cloud Run service to handle sudden traffic spikes efficiently without wasting resources during low traffic. Which combination of settings is best?

hard

A. Set concurrency to a moderate number (e.g., 10) and allow max instances to scale high

B. Set concurrency to 1 and max instances to 1

C. Set concurrency very high (e.g., 100) and max instances low (e.g., 2)

D. Disable concurrency and fix max instances to 10

Solution

1. Step 1: Balance concurrency and scaling for efficiency

   Moderate concurrency lets each instance handle multiple requests, saving resources.

2. Step 2: Allow max instances to scale high for spikes

   High max instances lets the service add instances when traffic spikes, keeping performance.

3. Final Answer:

   Set concurrency to a moderate number (e.g., 10) and allow max instances to scale high -> Option A

4. Quick Check:

   Moderate concurrency + high max instances = efficient scaling [OK]

Hint: Moderate concurrency + high max instances for spikes [OK]

Common Mistakes:

• Setting concurrency too low wastes instances
• Setting concurrency too high risks crashes
• Fixing max instances limits scaling