Thread pools in Operating Systems - Time & Space Complexity

Time Complexity: Thread pools

O(n)

Understanding Time Complexity

When using thread pools, it is important to understand how the time to complete tasks changes as the number of tasks grows.

We want to know how the total work time scales when many tasks are handled by a fixed number of threads.

Scenario Under Consideration

Analyze the time complexity of the following thread pool task execution.

ThreadPool pool = new ThreadPool(4); // 4 worker threads
for (int i = 0; i < n; i++) {
    pool.submit(() -> {
        performTask(); // each task takes constant time
    });
}
pool.waitForAllTasks();
    

This code submits n tasks to a thread pool with 4 threads, each task taking the same fixed time.

Identify Repeating Operations

Look for repeated actions that affect total time.

• Primary operation: Executing each of the n tasks.
• How many times: Each task runs once, total n tasks.
• The thread pool runs up to 4 tasks at the same time, but all n tasks must complete.

How Execution Grows With Input

As the number of tasks n increases, total work grows linearly because each task takes constant time.

Input Size (n)	Approx. Operations (task executions)
10	10 tasks executed
100	100 tasks executed
1000	1000 tasks executed

Pattern observation: Doubling the number of tasks roughly doubles the total work time, even with parallel threads.

Final Time Complexity

Time Complexity: O(n)

This means the total time grows directly in proportion to the number of tasks, despite running some in parallel.

Common Mistake

[X] Wrong: "Using a thread pool with multiple threads makes the total time constant regardless of tasks."

[OK] Correct: Even with multiple threads, all tasks must run, so total time still grows with the number of tasks, just faster than one thread.

Interview Connect

Understanding how thread pools affect time helps you explain real-world program performance and shows you can reason about parallel work efficiently.

Self-Check

What if the thread pool size increased with the number of tasks? How would the time complexity change?

Practice

1. What is the main purpose of a thread pool in an operating system?

easy

A. To create a new thread for every task without limit

B. To store data permanently on disk

C. To reuse a fixed number of threads to run multiple tasks efficiently

D. To manage memory allocation for processes

Solution

1. Step 1: Understand thread pool concept

   A thread pool manages a fixed number of threads to handle many tasks efficiently without creating new threads each time.

2. Step 2: Compare options with thread pool purpose

   Only To reuse a fixed number of threads to run multiple tasks efficiently correctly describes reusing threads for multiple tasks. Other options describe unrelated concepts.

3. Final Answer:

   To reuse a fixed number of threads to run multiple tasks efficiently -> Option C

4. Quick Check:

   Thread pool purpose = reuse threads [OK]

Hint: Thread pools reuse threads, not create new ones each time [OK]

Common Mistakes:

• Thinking thread pools create unlimited threads
• Confusing thread pools with memory management
• Assuming thread pools store data permanently

2. Which of the following is the correct way to describe how tasks are handled in a thread pool?

easy

A. Tasks run only one at a time sequentially

B. Tasks are executed immediately without waiting

C. Tasks are discarded if no thread is free

D. Tasks wait in a queue if all threads are busy

Solution

1. Step 1: Recall thread pool task management

   When all threads are busy, new tasks wait in a queue until a thread becomes free.

2. Step 2: Evaluate options against this behavior

   Tasks wait in a queue if all threads are busy correctly states tasks wait in a queue. Other options are incorrect because tasks are not discarded or run sequentially only.

3. Final Answer:

   Tasks wait in a queue if all threads are busy -> Option D

4. Quick Check:

   Task queueing = waiting tasks [OK]

Hint: Tasks queue up when threads are busy, not discarded [OK]

Common Mistakes:

• Believing tasks run immediately always
• Thinking tasks get dropped if no thread is free
• Assuming tasks run strictly one by one

3. Consider a thread pool with 3 threads. If 5 tasks are submitted at once, how many tasks will be running simultaneously?

medium

A. 3

B. 2

C. 5

D. 0

Solution

1. Step 1: Understand thread pool capacity

   The thread pool has 3 threads, so it can run up to 3 tasks at the same time.

2. Step 2: Analyze task submission

   When 5 tasks are submitted, 3 run immediately (one per thread), and 2 wait in the queue.

3. Final Answer:

   3 -> Option A

4. Quick Check:

   Threads limit running tasks = 3 [OK]

Hint: Running tasks = number of threads in pool [OK]

Common Mistakes:

• Assuming all tasks run simultaneously regardless of thread count
• Confusing queued tasks with running tasks
• Thinking no tasks run if more than threads

4. A developer notices that tasks submitted to a thread pool never start running and remain queued indefinitely. What is the most likely cause?

medium

A. The thread pool size is set to zero

B. Tasks are too short to run

C. The queue is empty

D. The CPU is overloaded

Solution

1. Step 1: Analyze thread pool size effect

   If the thread pool size is zero, no threads exist to run tasks, so tasks remain queued forever.

2. Step 2: Evaluate other options

   Tasks being short or queue empty do not cause indefinite waiting. CPU overload may slow but not block all tasks.

3. Final Answer:

   The thread pool size is set to zero -> Option A

4. Quick Check:

   Zero threads means no task execution [OK]

Hint: Zero threads means no tasks run, causing indefinite queue [OK]

Common Mistakes:

• Assuming short tasks cause waiting
• Thinking empty queue causes waiting
• Blaming CPU overload for all tasks stuck

5. You need to design a thread pool for a server that handles 100 simultaneous client requests. Which approach best balances resource use and performance?

hard

A. Create a thread pool with 100 threads to handle all requests at once

B. Create a thread pool with a fixed smaller number of threads (e.g., 10) and queue extra requests

C. Create a new thread for each request without pooling

D. Use a single thread to handle all requests sequentially

Solution

1. Step 1: Consider resource limits

   Creating 100 threads can exhaust system resources and reduce performance.

2. Step 2: Evaluate thread pool with queuing

   A fixed smaller thread pool (like 10 threads) efficiently reuses threads and queues extra requests, balancing load and resources.

3. Step 3: Reject other options

   Creating new threads per request wastes resources; single thread causes slow sequential handling.

4. Final Answer:

   Create a thread pool with a fixed smaller number of threads (e.g., 10) and queue extra requests -> Option B

5. Quick Check:

   Fixed small pool + queue = balanced performance [OK]

Hint: Use fewer threads than requests, queue extras for efficiency [OK]

Common Mistakes:

• Making thread pool size equal to requests
• Creating new thread per request wastes resources
• Using single thread causes slow processing