Concurrency considerations in LLD - Scalability & System Analysis

Scalability Analysis - Concurrency considerations

Growth Table: Concurrency Considerations

Users	Concurrent Requests	System Behavior	Concurrency Challenges
100	~50-100	Single server handles requests smoothly	Minimal locking, simple thread management
10,000	~5,000-10,000	Multiple threads/processes needed, some contention	Lock contention, race conditions start to appear
1,000,000	~500,000-1,000,000	Multiple servers, distributed concurrency control	Distributed locks, deadlocks, consistency issues
100,000,000	~50,000,000-100,000,000	Massive distributed systems, microservices	Complex coordination, eventual consistency, partition tolerance

First Bottleneck

The first bottleneck in concurrency is usually the lock contention on shared resources. As concurrent requests increase, threads or processes compete to access the same data or code sections, causing delays and reduced throughput.

Scaling Solutions

• Reduce Lock Scope: Minimize the code section under locks to reduce waiting time.
• Use Lock-Free Data Structures: Employ atomic operations and concurrent collections to avoid locks.
• Sharding: Partition data to reduce contention by isolating concurrent access.
• Horizontal Scaling: Add more servers to distribute load and concurrency.
• Optimistic Concurrency Control: Allow concurrent access and resolve conflicts after.
• Queue Requests: Serialize access to critical sections using message queues.

Back-of-Envelope Cost Analysis

Assuming 1,000 concurrent requests per server, a system with 10,000 concurrent requests needs ~10 servers.

Each lock contention adds latency; reducing lock time by 10ms saves 100 seconds cumulatively per 10,000 requests.

Network bandwidth and CPU usage increase with concurrency; plan for CPU cores and network capacity accordingly.

Interview Tip

When discussing concurrency, start by identifying shared resources and potential contention points. Explain how locks or synchronization work and their costs. Then discuss strategies to reduce contention and improve throughput, such as lock-free designs, sharding, or horizontal scaling.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Add read replicas and implement caching to reduce load on the primary database and handle increased concurrency without excessive locking.

Key Result

Concurrency bottlenecks start with lock contention on shared resources; reducing lock scope and horizontal scaling are key to handling growth.

Practice

1. What is the main purpose of using locks in concurrent systems?

easy

A. To allow unlimited access to shared resources

B. To prevent multiple threads from accessing shared data simultaneously

C. To speed up the execution of a single thread

D. To reduce memory usage in the system

Solution

1. Step 1: Understand concurrency risks

   When multiple threads access shared data at the same time, it can cause errors or inconsistent results.

2. Step 2: Role of locks

   Locks ensure only one thread accesses the shared data at a time, preventing conflicts and data corruption.

3. Final Answer:

   To prevent multiple threads from accessing shared data simultaneously -> Option B

4. Quick Check:

   Locks protect shared data = C [OK]

Hint: Locks protect shared data from simultaneous access [OK]

Common Mistakes:

• Thinking locks speed up single-thread execution
• Believing locks allow unlimited resource access
• Confusing locks with memory optimization

2. Which of the following is the correct way to acquire a lock in a typical low-level design?

easy

A. lock.notify() before accessing shared data

B. lock.release() before accessing shared data

C. lock.wait() after accessing shared data

D. lock.acquire() before accessing shared data

Solution

1. Step 1: Understand lock usage order

   To safely access shared data, a thread must first acquire the lock to block others.

2. Step 2: Correct method to acquire lock

   The method lock.acquire() is used to obtain the lock before accessing shared data.

3. Final Answer:

   lock.acquire() before accessing shared data -> Option D

4. Quick Check:

   Acquire lock first = A [OK]

Hint: Acquire lock before shared data access [OK]

Common Mistakes:

• Releasing lock before access
• Using wait or notify incorrectly
• Confusing acquire with release

3. Consider this pseudocode for two threads incrementing a shared counter without locks:

Thread 1: temp = counter
          temp = temp + 1
          counter = temp

Thread 2: temp = counter
          temp = temp + 1
          counter = temp

What is the possible final value of counter if it starts at 0?

medium

A. 2

B. Any negative number

C. 1

D. 0

Solution

1. Step 1: Analyze concurrent increments without locks

   Both threads read the same initial value 0, increment it to 1, and write back 1, causing one increment to be lost.

2. Step 2: Determine final counter value

   Because of race condition, the counter may only increase once, resulting in final value 1 instead of 2.

3. Final Answer:

   1 -> Option C

4. Quick Check:

   Race condition causes lost update = 1 [OK]

Hint: Without locks, increments can overwrite each other [OK]

Common Mistakes:

• Assuming both increments always succeed
• Ignoring race conditions
• Thinking counter can be negative here

4. In the following code snippet, what is the main concurrency issue?

lock.acquire()
shared_data.append(1)
# Missing lock.release()

medium

A. Deadlock due to missing lock release

B. Data race on shared_data

C. Syntax error in lock usage

D. No issue, code is safe

Solution

1. Step 1: Identify missing lock release

   The code acquires a lock but never releases it, so other threads waiting for the lock will block forever.

2. Step 2: Understand deadlock impact

   This causes a deadlock where threads cannot proceed, halting system progress.

3. Final Answer:

   Deadlock due to missing lock release -> Option A

4. Quick Check:

   Missing release causes deadlock = A [OK]

Hint: Always release locks after acquiring [OK]

Common Mistakes:

• Thinking it's a syntax error
• Assuming no issue without release
• Confusing deadlock with data race

5. You design a system where multiple threads read and write a shared cache. To improve performance, you want to allow multiple readers but only one writer at a time. Which concurrency control mechanism fits best?

hard

A. Use a read-write lock allowing concurrent reads but exclusive writes

B. Use a simple mutex lock for all access

C. Use no locks and rely on thread scheduling

D. Use a semaphore with count 1 for all operations

Solution

1. Step 1: Understand concurrency needs for readers and writers

   Multiple readers can safely access shared data simultaneously, but writers need exclusive access to avoid conflicts.

2. Step 2: Choose appropriate lock type

   A read-write lock allows many readers at once but only one writer, balancing concurrency and safety efficiently.

3. Final Answer:

   Use a read-write lock allowing concurrent reads but exclusive writes -> Option A

4. Quick Check:

   Read-write lock fits multiple readers, single writer = B [OK]

Hint: Read-write locks allow many readers, one writer [OK]

Common Mistakes:

• Using simple mutex reduces concurrency
• Ignoring need for exclusive write access
• Relying on no locks causes data races