Payment handling in LLD - Scalability & System Analysis

Scalability Analysis - Payment handling

Growth Table: Payment Handling System

Scale	Users	Transactions per Second (TPS)	Data Storage	Latency Requirements	System Changes
Small	100	1-5 TPS	Few MBs (transaction logs)	~1-2 seconds	Single app server, single DB instance, basic logging
Medium	10,000	100-500 TPS	GBs (transaction history, user data)	<1 second	Load balancer, multiple app servers, DB read replicas, caching
Large	1,000,000	5,000-10,000 TPS	TBs (full transaction history, audit logs)	<500 ms	Sharded DB, distributed cache, message queues, microservices
Very Large	100,000,000	100,000+ TPS	Petabytes (archival storage, compliance data)	<200 ms	Multi-region deployment, event-driven architecture, advanced fraud detection, CDN for static content

First Bottleneck

At small to medium scale, the database is the first bottleneck. It struggles to handle increasing transaction writes and reads, especially with ACID compliance and consistency requirements.

As TPS grows, the application servers may also become CPU and memory constrained due to encryption, validation, and communication with payment gateways.

At very large scale, network bandwidth and data partitioning challenges arise, especially for cross-region consistency and compliance.

Scaling Solutions

• Database scaling: Use read replicas to offload reads, implement sharding to distribute writes, and use connection pooling.
• Caching: Cache non-sensitive data like exchange rates or user preferences to reduce DB load.
• Horizontal scaling: Add more application servers behind a load balancer to handle more concurrent payment requests.
• Message queues: Use asynchronous processing for non-critical tasks like notifications or reporting to reduce latency.
• Microservices: Separate payment processing, fraud detection, and user management into services for independent scaling.
• Network and multi-region: Deploy services closer to users and use CDNs for static content to reduce latency.
• Security and compliance: Use encryption, tokenization, and PCI DSS compliant services to safely handle payment data.

Back-of-Envelope Cost Analysis

• At 10,000 TPS, expect ~864 million transactions/day.
• Each transaction record ~1 KB -> ~864 GB/day storage needed before compression or archival.
• Network bandwidth: 10,000 TPS * 1 KB = ~10 MB/s sustained, plus overhead.
• Application servers: Each handles ~2,000 concurrent connections, so 5-10 servers needed at medium scale.
• Database: Single instance handles ~5,000 QPS; need replicas and sharding beyond that.
• Cloud costs scale with storage, compute, and network usage; optimize with caching and archiving.

Interview Tip

Start by clarifying the expected scale and latency requirements.

Identify the critical components: payment gateway, database, app servers.

Discuss bottlenecks at each scale and propose targeted solutions.

Highlight security and compliance as non-negotiable constraints.

Use real numbers to justify your scaling choices.

Self-Check Question

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first and why?

Answer: Add read replicas to distribute read load and implement connection pooling. For writes, consider sharding or partitioning to distribute write load. This addresses the database bottleneck before scaling app servers.

Key Result

The database is the first bottleneck in payment handling systems as transaction volume grows. Scaling requires read replicas, sharding, caching, and horizontal app server scaling while maintaining security and compliance.

Practice

1. What is the primary purpose of a payment handling system in software design?

easy

A. To store user profile pictures

B. To securely process and record financial transactions

C. To manage user login and authentication

D. To display product information to users

Solution

1. Step 1: Understand the role of payment handling

   Payment handling systems focus on managing money transfers safely and reliably.

2. Step 2: Identify the core function

   The core function is to process payments securely and keep records of transactions.

3. Final Answer:

   To securely process and record financial transactions -> Option B

4. Quick Check:

   Payment handling = Secure transaction processing [OK]

Hint: Payment handling means safe money transfer and record keeping [OK]

Common Mistakes:

• Confusing payment handling with user authentication
• Thinking it manages product display
• Assuming it stores user media files

2. Which of the following is the correct sequence of steps in a typical payment processing flow?

easy

A. Notify user -> Record transaction -> Process payment -> Validate payment details

B. Record transaction -> Validate payment details -> Notify user -> Process payment

C. Validate payment details -> Process payment -> Record transaction -> Notify user

D. Process payment -> Notify user -> Validate payment details -> Record transaction

Solution

1. Step 1: Identify logical payment flow order

   First, payment details must be validated to ensure correctness.

2. Step 2: Follow with processing, recording, and notifying

   After validation, payment is processed, transaction recorded, then user notified.

3. Final Answer:

   Validate payment details -> Process payment -> Record transaction -> Notify user -> Option C

4. Quick Check:

   Payment flow = Validate -> Process -> Record -> Notify [OK]

Hint: Payment flows from validation to processing, then record and notify [OK]

Common Mistakes:

• Not validating before processing
• Not recording transaction before notifying
• Mixing notification before processing

3. Consider this simplified payment processing pseudocode:

def process_payment(amount, card_info):
    if not validate_card(card_info):
        return "Invalid card"
    if amount <= 0:
        return "Invalid amount"
    if not charge_card(card_info, amount):
        return "Charge failed"
    record_transaction(card_info, amount)
    return "Payment successful"

What will be the output of process_payment(100, 'expired_card') if validate_card returns False for expired cards?

medium

A. "Invalid card"

B. "Charge failed"

C. "Payment successful"

D. "Invalid amount"

Solution

1. Step 1: Check card validation result

   Since validate_card returns False for expired cards, the first if condition triggers.

2. Step 2: Return error message immediately

   The function returns "Invalid card" without further processing.

3. Final Answer:

   "Invalid card" -> Option A

4. Quick Check:

   Expired card -> validate_card = False -> "Invalid card" [OK]

Hint: If validation fails, function returns error immediately [OK]

Common Mistakes:

• Assuming charge_card runs despite invalid card
• Confusing invalid amount with invalid card
• Expecting success despite validation failure

4. A payment system logs duplicate transactions when retrying failed payments. Which design fix will best prevent this issue?

medium

A. Use unique transaction IDs and check before recording

B. Increase payment timeout duration

C. Remove transaction logging entirely

D. Allow multiple retries without checks

Solution

1. Step 1: Identify cause of duplicate logs

   Retries cause repeated transaction records without uniqueness checks.

2. Step 2: Implement unique transaction IDs and check

   Assign unique IDs and verify before logging to avoid duplicates.

3. Final Answer:

   Use unique transaction IDs and check before recording -> Option A

4. Quick Check:

   Unique IDs prevent duplicate transaction logs [OK]

Hint: Unique IDs stop duplicate transaction records [OK]

Common Mistakes:

• Ignoring duplicate checks on retries
• Removing logging which loses audit trail
• Increasing timeout doesn't fix duplicates

5. You are designing a payment system that must handle 10,000 transactions per second with minimal latency and high reliability. Which architectural approach best supports this requirement?

hard

A. Store all payment requests in a single database table and process sequentially

B. Process all payments synchronously on a single server to ensure order

C. Use client-side scripts to process payments directly without server validation

D. Use a distributed message queue to process payments asynchronously with multiple worker nodes

Solution

1. Step 1: Analyze scalability and latency needs

   Handling 10,000 TPS requires distributing load and minimizing blocking.

2. Step 2: Choose asynchronous distributed processing

   Using a message queue with multiple workers allows parallel processing and reliability.

3. Step 3: Eliminate options causing bottlenecks or insecurity

   Single server or sequential DB processing causes bottlenecks; client-side processing lacks security.

4. Final Answer:

   Use a distributed message queue to process payments asynchronously with multiple worker nodes -> Option D

5. Quick Check:

   High TPS + low latency = distributed async processing [OK]

Hint: Distribute load with async workers for high TPS [OK]

Common Mistakes:

• Using single server causing bottlenecks
• Sequential DB processing slowing throughput
• Relying on client-side payment processing