Concurrency and scaling in GCP - Time & Space Complexity

Time Complexity: Concurrency and scaling

O(n)

Understanding Time Complexity

When systems handle many tasks at once, understanding how time grows with more work helps us plan better.

We want to know how adding more users or requests affects the time to complete tasks.

Scenario Under Consideration

Analyze the time complexity of scaling a service with concurrent requests.

// Pseudocode for handling concurrent requests
for each request in incoming_requests:
  start new thread to process request
  process request asynchronously
  return response when done

This sequence shows how a service handles multiple requests at the same time using concurrency.

Identify Repeating Operations

Look at what repeats as requests increase.

• Primary operation: Starting a new thread or process for each request.
• How many times: Once per incoming request.

How Execution Grows With Input

Each new request adds one more concurrent operation.

Input Size (n)	Approx. API Calls/Operations
10	10 concurrent threads/processes
100	100 concurrent threads/processes
1000	1000 concurrent threads/processes

Pattern observation: The number of concurrent operations grows directly with the number of requests.

Final Time Complexity

Time Complexity: O(n)

This means the time to start handling all requests grows linearly as more requests come in.

Common Mistake

[X] Wrong: "Adding more requests won't affect time because they run at the same time."

[OK] Correct: Even if requests run concurrently, starting and managing each one takes time and resources, so total work grows with requests.

Interview Connect

Understanding how concurrency scales helps you design systems that handle many users smoothly, a key skill in cloud roles.

Self-Check

"What if we used a fixed number of worker threads instead of one per request? How would the time complexity change?"