The CAP theorem in HLD - Scalability & System Analysis

Scalability Analysis - The CAP theorem

Growth Table: The CAP Theorem at Different Scales

Scale	Users	Data Centers	Network Latency	Consistency	Availability	Partition Tolerance
Small	100 users	1	Low	Strong consistency easy	High availability easy	Minimal partitions
Medium	10K users	2-3	Moderate	Trade-offs start	Maintained with retries	Partitions possible
Large	1M users	Multiple geo-distributed	High	Often eventual consistency	High availability prioritized	Partitions common
Very Large	100M users	Global multi-region	Very high	Mostly eventual consistency	Availability critical	Partition tolerance mandatory

First Bottleneck

As the system grows, network partitions become inevitable due to unreliable communication between data centers. The first bottleneck is the trade-off between consistency and availability during partitions. You cannot have both strong consistency and full availability when partitions happen.

Scaling Solutions

• Choose your priority: Decide if your system needs strong consistency (CP) or high availability (AP) during partitions.
• Use replication: Replicate data across nodes to improve availability and fault tolerance.
• Implement eventual consistency: Accept temporary inconsistencies to keep the system available.
• Partition tolerance: Design the system to handle network failures gracefully.
• Use consensus algorithms: For CP systems, use protocols like Paxos or Raft to maintain consistency.
• Client-side conflict resolution: For AP systems, resolve conflicts after partitions heal.

Back-of-Envelope Cost Analysis

Assuming 1M users with 1 request per second:

• Requests per second: ~1,000,000 QPS
• Single database node handles: ~5,000 QPS → Need ~200 nodes or sharding
• Network bandwidth: 1 Gbps = 125 MB/s, so multiple data centers need high bandwidth links
• Storage: Depends on data size; replication increases storage needs by number of replicas
• Latency: Higher with geo-distribution, affects consistency choices

Interview Tip

When discussing CAP theorem in an interview, start by explaining the three guarantees clearly. Then, describe real-world scenarios where you would prioritize consistency over availability or vice versa. Use examples like banking (CP) vs social media feeds (AP). Finally, mention how network partitions force trade-offs and how you would design your system accordingly.

Self Check

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add read replicas and implement caching to reduce load on the primary database. Consider sharding data to distribute load. Also, evaluate if consistency requirements allow for eventual consistency to improve availability.

Key Result

The CAP theorem shows that as systems scale and network partitions occur, you must choose between consistency and availability; partition tolerance is mandatory at large scale.

Practice

1. Which two properties can a distributed system guarantee at the same time according to the CAP theorem?

easy

A. Consistency and Availability

B. Availability and Partition tolerance

C. Consistency and Partition tolerance

D. Consistency, Availability, and Partition tolerance all together

Solution

1. Step 1: Understand the CAP theorem basics

   The CAP theorem states that a distributed system can only guarantee two out of three properties: Consistency, Availability, and Partition tolerance.

2. Step 2: Identify which pairs are possible

   Systems can guarantee Availability and Partition tolerance or Consistency and Partition tolerance, but not all three simultaneously. However, only Availability and Partition tolerance can be guaranteed simultaneously in the presence of network partitions.

3. Final Answer:

   Availability and Partition tolerance -> Option B

4. Quick Check:

   CAP theorem = two properties only [OK]

Hint: Remember CAP means pick only two properties [OK]

Common Mistakes:

• Thinking all three properties can be guaranteed simultaneously
• Confusing Availability with Partition tolerance
• Ignoring network partitions in distributed systems

2. Which of the following is the correct definition of Partition tolerance in the CAP theorem?

easy

A. The system can handle unlimited data growth

B. All nodes always return the same data at the same time

C. The system is always available to respond to requests

D. The system continues to operate despite network failures between nodes

Solution

1. Step 1: Define Partition tolerance

   Partition tolerance means the system keeps working even if some network connections between nodes fail or messages are lost.

2. Step 2: Match definitions to options

   The system continues to operate despite network failures between nodes correctly describes Partition tolerance as handling network failures. Other options describe consistency, availability, or scalability.

3. Final Answer:

   The system continues to operate despite network failures between nodes -> Option D

4. Quick Check:

   Partition tolerance = handle network splits [OK]

Hint: Partition tolerance means surviving network splits [OK]

Common Mistakes:

• Confusing Partition tolerance with Availability
• Thinking Partition tolerance means data consistency
• Assuming Partition tolerance means unlimited scaling

3. Consider a distributed database that chooses Availability and Partition tolerance over Consistency. What is the expected behavior during a network partition?

medium

A. The system may return stale or inconsistent data but remains available

B. The system rejects all writes until the partition is resolved

C. The system becomes unavailable to maintain consistency

D. The system automatically merges all data without conflicts

Solution

1. Step 1: Identify chosen CAP properties

   The system chooses Availability and Partition tolerance, sacrificing Consistency.

2. Step 2: Analyze behavior during network partition

   Because it prioritizes Availability, the system stays up and responds to requests, but data may be inconsistent or stale.

3. Final Answer:

   The system may return stale or inconsistent data but remains available -> Option A

4. Quick Check:

   Availability + Partition tolerance = possible inconsistency [OK]

Hint: Availability + Partition tolerance means possible stale data [OK]

Common Mistakes:

• Assuming system rejects writes to keep consistency
• Thinking system becomes unavailable during partition
• Believing automatic conflict-free merges always happen

4. A developer claims their distributed system guarantees Consistency, Availability, and Partition tolerance simultaneously. What is the most likely issue?

medium

A. The system is using eventual consistency, not strong consistency

B. The system is only tested on a single node

C. The system is ignoring network partitions or failures

D. The system uses synchronous replication

Solution

1. Step 1: Recall CAP theorem limitation

   The CAP theorem states all three properties cannot be guaranteed simultaneously in a distributed system.

2. Step 2: Identify the likely mistake

   If the system claims all three, it probably ignores or does not handle network partitions properly, which is unrealistic in real distributed environments.

3. Final Answer:

   The system is ignoring network partitions or failures -> Option C

4. Quick Check:

   CAP theorem forbids all three together [OK]

Hint: All three CAP properties together is impossible [OK]

Common Mistakes:

• Assuming eventual consistency means full consistency
• Thinking synchronous replication solves CAP limits
• Ignoring real network failures in testing

5. You are designing a global online banking system where transactions must be accurate and consistent even during network failures. According to the CAP theorem, which two properties should you prioritize and why?

hard

A. Consistency and Partition tolerance, to maintain accurate data despite network partitions, even if some requests are delayed

B. Availability and Partition tolerance, to keep the system running even if data is temporarily inconsistent

C. All three properties equally, because banking requires perfect reliability

D. Consistency and Availability, to ensure all users see the same data and system is always responsive

Solution

1. Step 1: Understand banking system requirements

   Banking needs accurate, consistent transactions even if network issues occur, so data correctness is critical.

2. Step 2: Apply CAP theorem to design choice

   To keep data consistent during network partitions, the system must prioritize Consistency and Partition tolerance, accepting some availability delays.

3. Final Answer:

   Consistency and Partition tolerance, to maintain accurate data despite network partitions, even if some requests are delayed -> Option A

4. Quick Check:

   Banking = Consistency + Partition tolerance [OK]

Hint: Banking needs consistency over availability during partitions [OK]

Common Mistakes:

• Choosing availability over consistency for critical data
• Assuming all three CAP properties can be met
• Ignoring impact of network partitions on data accuracy