Idempotency for safe retries in HLD - System Design Exercise

Design: Idempotent Request Handling System

Design focuses on server-side idempotency handling for API requests. Client-side retry logic and network error handling are out of scope.

Functional Requirements

FR1: Allow clients to safely retry requests without causing duplicate effects

FR2: Ensure that repeated identical requests produce the same result as a single request

FR3: Support high throughput with minimal latency impact

FR4: Handle concurrent retries gracefully

FR5: Provide clear feedback to clients about request status

Non-Functional Requirements

NFR1: System must handle up to 10,000 concurrent requests with retries

NFR2: API response latency p99 should be under 200ms including idempotency checks

NFR3: Availability target of 99.9% uptime

NFR4: Idempotency keys must expire after 24 hours to limit storage

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

API Gateway or Load Balancer

Idempotency Key Store (cache or database)

Application Server with idempotency logic

Persistent Database for final data storage

Response Cache for repeated requests

Design Patterns

Idempotency Key Pattern

Cache Aside Pattern

Write-Ahead Logging

Optimistic Concurrency Control

Reference Architecture

Client
  |
  | HTTP Request with Idempotency-Key header
  v
API Gateway / Load Balancer
  |
  | Forward request
  v
Application Server
  |
  | Check Idempotency Key Store
  |-- If key exists and completed: return stored response
  |-- If key exists and in progress: wait or reject
  |-- If key missing: process request
  |
  | Process business logic
  |
  | Store response and mark key as completed
  v
Persistent Database

Idempotency Key Store (Redis or DB) connected to Application Server

Components

API Gateway / Load Balancer

Nginx / AWS ALB

Route incoming requests and forward Idempotency-Key header

Idempotency Key Store

Redis or NoSQL DB

Store request states and responses keyed by Idempotency-Key

Application Server

Node.js / Java / Python

Check idempotency keys, process requests, store results

Persistent Database

PostgreSQL / MySQL

Store final business data and transaction records

Request Flow

1. Client sends request with unique Idempotency-Key header

2. API Gateway forwards request to Application Server

3. Application Server checks Idempotency Key Store for the key

4. If key exists and request completed, return stored response immediately

5. If key exists and request in progress, reject or wait to avoid duplicate processing

6. If key does not exist, mark key as in-progress and process request

7. After processing, store response and mark key as completed in Idempotency Key Store

8. Return response to client

9. Idempotency keys expire after 24 hours to free storage

Database Schema

Entities: - IdempotencyKey: key (string, PK), status (enum: in-progress, completed), response_data (json), created_at (timestamp), expires_at (timestamp) - BusinessEntity: id (PK), data fields... Relationships: - IdempotencyKey is independent but linked logically to BusinessEntity by request context

Scaling Discussion

Bottlenecks

Idempotency Key Store can become a hotspot under high concurrency

Application Server CPU and memory usage due to waiting or locking on keys

Database write latency affecting overall request processing time

Storage growth for idempotency keys over time

Solutions

Use a distributed in-memory cache like Redis Cluster for Idempotency Key Store to scale horizontally

Implement optimistic concurrency control or locking with timeouts to handle concurrent retries

Batch writes or use asynchronous processing to reduce database latency impact

Set TTL (time-to-live) on idempotency keys to automatically expire and clean up storage

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why idempotency is important for safe retries and user experience

Describe how idempotency keys prevent duplicate processing

Discuss storage choices and expiration strategies for keys

Highlight concurrency challenges and how to handle them

Mention latency and availability targets and how design meets them

Practice

1. What is the main purpose of idempotency in system design for safe retries?

easy

A. To increase the number of retries allowed

B. To ensure repeated requests produce the same result without side effects

C. To speed up the processing of new requests

D. To encrypt data during transmission

Solution

1. Step 1: Understand idempotency concept

   Idempotency means that making the same request multiple times results in the same outcome without causing additional changes.

2. Step 2: Identify the purpose in retries

   In safe retries, idempotency prevents duplicate processing and side effects by returning the same result for repeated requests.

3. Final Answer:

   To ensure repeated requests produce the same result without side effects -> Option B

4. Quick Check:

   Idempotency = Same result, no side effects [OK]

Hint: Idempotency means retrying won't change the outcome [OK]

Common Mistakes:

• Confusing idempotency with performance optimization
• Thinking idempotency increases retry limits
• Assuming idempotency encrypts data

2. Which of the following is the correct way to implement idempotency in an API request?

easy

A. Retry the request without any tracking

B. Encrypt the request payload for security

C. Ignore request IDs and process every request as new

D. Use a unique request ID to detect duplicates and avoid reprocessing

Solution

1. Step 1: Identify idempotency implementation method

   Idempotency requires tracking requests to detect duplicates, typically using a unique request ID.

2. Step 2: Evaluate options

   Only using a unique request ID allows the system to recognize repeated requests and avoid reprocessing.

3. Final Answer:

   Use a unique request ID to detect duplicates and avoid reprocessing -> Option D

4. Quick Check:

   Unique request ID = Idempotency key [OK]

Hint: Unique request ID prevents duplicate processing [OK]

Common Mistakes:

• Retrying blindly without tracking
• Ignoring request IDs
• Confusing encryption with idempotency

3. Consider a system that stores results of processed requests keyed by a unique request ID. If a client sends the same request twice with the same ID, what will the system return on the second request?

medium

A. A new result generated by reprocessing

B. An error indicating duplicate request

C. The original result from the first request

D. A timeout response

Solution

1. Step 1: Understand stored result usage

   The system caches the result of the first request using the unique request ID.

2. Step 2: Determine response for repeated request

   When the same request ID is received again, the system returns the cached original result to maintain idempotency.

3. Final Answer:

   The original result from the first request -> Option C

4. Quick Check:

   Repeated request returns cached result [OK]

Hint: Same ID returns cached result, not reprocessed output [OK]

Common Mistakes:

• Expecting an error on duplicate requests
• Assuming reprocessing happens every time
• Confusing timeout with duplicate handling

4. A developer implemented idempotency by storing request IDs but forgot to store the response results. What problem will this cause on retries?

medium

A. The system will reprocess the request causing side effects

B. The system will return the original result correctly

C. The system will reject all requests

D. The system will crash due to missing data

Solution

1. Step 1: Analyze missing response storage impact

   Without storing the response, the system cannot return the original result on retries.

2. Step 2: Identify retry behavior

   Since the system lacks the cached result, it reprocesses the request, causing potential side effects and breaking idempotency.

3. Final Answer:

   The system will reprocess the request causing side effects -> Option A

4. Quick Check:

   Missing response storage = Reprocessing on retry [OK]

Hint: Store both request ID and response to avoid reprocessing [OK]

Common Mistakes:

• Assuming storing request ID alone is enough
• Expecting system to reject retries automatically
• Thinking system crashes without response data

5. You design a payment system that must be idempotent to handle retries safely. Which combination of techniques best ensures idempotency and prevents duplicate charges?

hard

A. Use unique request IDs, store payment results, and check before processing

B. Allow retries without tracking, rely on client to avoid duplicates

C. Encrypt payment data and retry immediately on failure

D. Process every request independently to ensure payment completion

Solution

1. Step 1: Identify idempotency requirements for payments

   Payments must not be charged multiple times on retries, so tracking requests and results is essential.

2. Step 2: Evaluate techniques for safe retries

   Using unique request IDs to detect duplicates, storing results to return on retries, and checking before processing prevents duplicate charges.

3. Final Answer:

   Use unique request IDs, store payment results, and check before processing -> Option A

4. Quick Check:

   Unique ID + stored result = Safe payment retries [OK]

Hint: Combine unique IDs and result storage for safe retries [OK]

Common Mistakes:

• Relying on client to avoid duplicates
• Ignoring result storage
• Processing every request without checks