Thread safety in design in LLD - System Design Exercise

Design: Thread Safe Shared Resource Manager

Design focuses on thread safety mechanisms for shared resource access in a multi-threaded environment. Out of scope are distributed system concerns and persistent storage design.

Functional Requirements

FR1: Allow multiple threads to access shared resources safely

FR2: Prevent race conditions and data corruption

FR3: Support concurrent read operations

FR4: Ensure exclusive access for write operations

FR5: Provide mechanisms to avoid deadlocks and starvation

Non-Functional Requirements

NFR1: Handle up to 100 concurrent threads

NFR2: API response latency under 50ms for read operations

NFR3: System availability of 99.9%

NFR4: Minimal performance overhead due to synchronization

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Mutexes or locks

Read-write locks

Atomic operations

Condition variables

Thread-safe data structures

Design Patterns

Locking patterns (coarse-grained vs fine-grained)

Immutable objects for safe sharing

Thread confinement

Lock-free and wait-free algorithms

Double-checked locking

Reference Architecture

  +---------------------+
  |  Client Threads     |
  |  (Concurrent Access)|
  +----------+----------+
             |
             v
  +---------------------+
  | Thread Safe Manager  |
  |  - Read-Write Lock   |
  |  - Mutexes           |
  |  - Atomic Counters   |
  +----------+----------+
             |
             v
  +---------------------+
  | Shared Resources     |
  |  (Protected Data)    |
  +---------------------+

Components

Client Threads

Any multi-threaded environment

Simulate concurrent access to shared resources

Thread Safe Manager

Read-Write Locks, Mutexes, Atomic Operations

Coordinate safe access to shared resources, allowing concurrent reads and exclusive writes

Shared Resources

In-memory data structures

Data or objects accessed concurrently by threads

Request Flow

1. 1. Client thread requests access to a shared resource.

2. 2. Thread Safe Manager checks request type (read or write).

3. 3. For read requests, acquire read lock allowing multiple concurrent readers.

4. 4. For write requests, acquire exclusive write lock blocking other readers and writers.

5. 5. Thread accesses or modifies the shared resource safely.

6. 6. Thread releases the lock after operation completes.

7. 7. Thread Safe Manager ensures no deadlocks by ordering lock acquisition and using timeouts if needed.

Database Schema

Not applicable as this design focuses on in-memory thread safety mechanisms.

Scaling Discussion

Bottlenecks

Lock contention when many threads try to write simultaneously

Performance degradation due to excessive locking overhead

Potential deadlocks if locks are not managed carefully

Starvation of writer threads if readers dominate

Solutions

Use fine-grained locking to reduce contention by locking smaller parts of data

Implement lock-free or wait-free algorithms where possible

Apply timeout and deadlock detection mechanisms

Use fair read-write locks to balance reader and writer access

Partition data to reduce shared resource hotspots

Interview Tips

Time: Spend 10 minutes understanding thread safety challenges and clarifying requirements, 20 minutes designing locking strategies and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why thread safety is critical to prevent data corruption

Discuss trade-offs between concurrency and locking overhead

Describe different locking mechanisms and when to use each

Highlight deadlock and starvation risks and mitigation strategies

Show understanding of scaling challenges and advanced concurrency patterns

Practice

1. What does thread safety in system design primarily ensure?

easy

A. Multiple threads can access shared data without causing errors

B. The system runs faster by using more threads

C. Only one thread runs at a time in the entire system

D. Threads do not use any shared resources

Solution

1. Step 1: Understand thread safety concept

   Thread safety means multiple threads can work with shared data without causing conflicts or errors.

2. Step 2: Analyze options

   Multiple threads can access shared data without causing errors correctly states this. Options B, C, and D misunderstand thread safety or describe unrelated concepts.

3. Final Answer:

   Multiple threads can access shared data without causing errors -> Option A

4. Quick Check:

   Thread safety = safe shared data access [OK]

Hint: Thread safety means safe shared data access [OK]

Common Mistakes:

• Confusing thread safety with performance
• Thinking only one thread runs at a time
• Assuming no shared data is used

2. Which of the following is the correct way to declare a lock object in a typical low-level design for thread safety?

easy

A. lock = synchronized()

B. lock = new Lock()

C. lock = create_lock()

D. lock = Lock()

Solution

1. Step 1: Identify common lock declaration syntax

   In many low-level designs, a lock is created by calling a constructor like Lock().

2. Step 2: Compare options

   lock = Lock() uses lock = Lock(), which is typical. lock = new Lock() uses 'new' which is not common in low-level design languages. lock = create_lock() and D use incorrect or non-standard functions.

3. Final Answer:

   lock = Lock() -> Option D

4. Quick Check:

   Lock creation = Lock() [OK]

Hint: Lock objects are usually created by calling Lock() [OK]

Common Mistakes:

• Using 'new' keyword incorrectly
• Assuming lock creation uses special functions
• Confusing lock with synchronization keyword

3. Consider this pseudocode for a shared counter increment:

lock.acquire()
counter = counter + 1
lock.release()
print(counter)

If two threads run this code simultaneously starting with counter = 0, what is the possible output?

medium

A. 0

B. 3

C. 2

D. Any number greater than 2

Solution

1. Step 1: Understand lock usage in code

   The lock ensures only one thread increments the counter at a time, preventing race conditions.

2. Step 2: Calculate final counter value

   Two threads each increment once, so counter goes from 0 to 2 safely.

3. Final Answer:

   2 -> Option C

4. Quick Check:

   Lock ensures increments are safe, so counter = 2 [OK]

Hint: Locks prevent lost updates, so increments add up [OK]

Common Mistakes:

• Ignoring lock and assuming race condition
• Thinking output can be 0 or 1 due to concurrency
• Assuming counter can exceed 2 without loops

4. In this code snippet, what is the main thread safety issue?

lock.acquire()
shared_list.append(1)
# Missing lock.release()

medium

A. No issue, code is safe

B. Deadlock due to missing lock release

C. Syntax error in lock usage

D. Race condition on shared_list

Solution

1. Step 1: Analyze lock usage

   The code acquires a lock but never releases it, causing other threads to wait forever.

2. Step 2: Identify consequence

   This causes a deadlock, where threads block indefinitely waiting for the lock.

3. Final Answer:

   Deadlock due to missing lock release -> Option B

4. Quick Check:

   Missing release = deadlock [OK]

Hint: Always release locks to avoid deadlocks [OK]

Common Mistakes:

• Thinking race condition occurs despite lock
• Assuming syntax error without checking code
• Believing code is safe without release

5. You design a system where multiple threads update a shared cache. To improve performance, you want to minimize locking time. Which design approach best balances thread safety and performance?

hard

A. Use fine-grained locks for each cache entry

B. Avoid locks and allow unsynchronized updates

C. Use a single global lock for all cache updates

D. Lock the entire cache for every read and write

Solution

1. Step 1: Understand locking strategies

   A single global lock (Use a single global lock for all cache updates) causes contention and slows performance. No locks (Avoid locks and allow unsynchronized updates) risks data corruption. Locking entire cache for reads and writes (Lock the entire cache for every read and write) is too heavy.

2. Step 2: Choose fine-grained locks

   Fine-grained locks (Use fine-grained locks for each cache entry) lock only parts of the cache, reducing waiting time and keeping thread safety.

3. Final Answer:

   Use fine-grained locks for each cache entry -> Option A

4. Quick Check:

   Fine-grained locks = safety + speed [OK]

Hint: Fine-grained locks reduce wait and keep safety [OK]

Common Mistakes:

• Using one big lock causing slowdowns
• Skipping locks causing data errors
• Locking too much causing bottlenecks