Bird
Raised Fist0

Idempotency in APIs - Why It Is Critical for Payment and Retry Systems

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/5
API Gateway

Receives payment request with idempotency key

Client sends POST /payment with unique idempotency key header

Protocol:HTTP REST
BeforeClient has payment data and idempotency key ready
AfterAPI Gateway forwards request to Idempotency Service
Idempotency Service

Checks Cassandra for existing idempotency key

Queries Cassandra DB to verify if request was processed

Protocol:Cassandra GET
BeforeIdempotency Service has incoming request with key
AfterDetermines if request is new or duplicate
Idempotency Service

Stores new idempotency key and enqueues payment request

Writes key to Cassandra and produces event to Kafka

Protocol:Cassandra PUT + Kafka Produce
BeforeRequest identified as new, no prior record found
AfterIdempotency key stored; payment event queued in Kafka
Message Queue (Kafka)

Holds payment event until consumed

Durably stores payment request event for processing

Protocol:Kafka Store
BeforePayment event produced by Idempotency Service
AfterEvent available for Payment Processing Service consumption
Payment Processing Service

Consumes event and processes payment exactly once

Executes payment logic and updates idempotency record

Protocol:Kafka Consume + Cassandra PUT
BeforePayment event pending in Kafka
AfterPayment processed; idempotency record marked complete
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────┐
│   API Gateway     │
└─────────┬─────────┘
          │ HTTP POST /payment (REST)
          ▼
┌───────────────────────┐
│  Idempotency Service   │
│  - Checks request key  │
│  - Stores idempotency  │
└─────────┬─────────────┘
          │ Cassandra GET/PUT idempotency key
          ▼
┌───────────────────────────┐
│  Cassandra DB             │
│  - Stores idempotency rec │
└─────────┬─────────────────┘
          │
          ▼
┌───────────────────────────┐
│  Idempotency Service      │
│  - If new request, enqueue│
│  - Else return cached res │
└─────────┬─────────────────┘
          │ Kafka Produce payment req
          ▼
┌───────────────────────────┐
│  Message Queue (Kafka)    │
│  - Durable event log      │
└─────────┬─────────────────┘
          │ Kafka Consume payment event
          ▼
┌───────────────────────────┐
│  Payment Processing Svc   │
│  - Processes payment once │
│  - Updates idempotency rec│
└───────────────────────────┘
Design Rationale

This architecture ensures idempotency in payment APIs by using a dedicated Idempotency Service that checks and stores unique request keys in Cassandra DB. Incoming payment requests pass through the API Gateway to the Idempotency Service, which queries Cassandra to detect duplicates. New requests are stored and enqueued into Kafka for asynchronous processing by the Payment Processing Service. Kafka guarantees durable event storage, preventing message loss during retries. The Payment Processing Service consumes events exactly once, processes payments, and updates the idempotency record to prevent duplicate charges. This design prevents double payments even if clients retry requests due to network timeouts, ensuring safe and reliable payment processing.

Primary Focus

✓

Practice

(1/5)
1. A system must ingest 1 million log events per second from thousands of servers with minimal latency and guarantee ordering per source. Which component is best suited to handle this ingestion reliably?
easy
A. Use a relational database with batch inserts every second
B. Use Redis as an in-memory queue for all logs
C. Use HTTP REST API calls directly to storage nodes
D. Use Kafka with partitioning keyed by server ID

Solution

  1. Step 1: Identify ingestion requirements

    High throughput (1M logs/sec) and ordering per source require a scalable, partitioned messaging system.

  2. Step 2: Evaluate Kafka's partitioning

    Kafka partitions keyed by server ID ensure ordering per source and handle high throughput with durability.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Kafka partitions keyed by source -> ordering and high throughput [check]
Hint: Partition by source for ordering and scale [check]
Common Mistakes:
2. Without a distributed consensus system (like Paxos or Raft) coordinating leader election, what exactly breaks in a system designed for 99.999% availability?
medium
A. Clients receive stale data but system continues operating normally
B. Multiple nodes may believe they're leader simultaneously, causing conflicting writes and data corruption
C. The system automatically fails over to a backup leader without downtime
D. Leader election happens instantly with no impact on availability

Solution

  1. Step 1: Role of distributed consensus in leader election

    Consensus protocols ensure exactly one leader at a time, preventing split-brain scenarios.

  2. Step 2: Consequences of missing consensus

    Without consensus, multiple nodes may act as leader simultaneously, causing conflicting writes and data corruption, breaking availability guarantees.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    No consensus -> split-brain -> conflicting writes -> data corruption [check]
Hint: No consensus -> split-brain -> conflicting writes break availability [check]
Common Mistakes:
3. If the distributed lock service is unavailable, what exactly breaks in the hotel booking system's reservation flow?
medium
A. Reservations proceed normally but double bookings increase due to lack of coordination
B. Reservation requests block indefinitely waiting for locks, causing user timeouts and failed bookings
C. Inventory service marks rooms as available incorrectly, causing UI to show wrong availability
D. Payment processing fails because it depends on the lock service for transaction ordering

Solution

  1. Step 1: Identify lock service role in reservation flow

    The lock service ensures exclusive access to room inventory during reservation to prevent double booking.

  2. Step 2: Trace failure impact

    Without the lock service, Inventory cannot reliably mark rooms as unavailable, leading to stale or incorrect availability states shown to users (Inventory service marks rooms as available incorrectly, causing UI to show wrong availability). Reservations may still proceed but with inconsistent inventory state. Blocking indefinitely (Reservation requests block indefinitely waiting for locks, causing user timeouts and failed bookings) is unlikely as fallback mechanisms exist. Payment processing (Payment processing fails because it depends on the lock service for transaction ordering) is independent of locking. Double bookings (Reservations proceed normally but double bookings increase due to lack of coordination) may increase but the immediate failure is incorrect inventory state.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Lock service failure causes stale inventory availability state [check]
Hint: Lock service failure -> stale inventory state, not immediate booking block [check]
Common Mistakes:
4. If the Kafka cluster responsible for photo upload event streaming goes down, what exactly breaks in the system?
medium
A. Photo uploads fail immediately with HTTP 500 errors to users
B. Feed generation continues normally but stories fail to update
C. Photo metadata extraction and thumbnail generation stop; uploads succeed but processing is delayed indefinitely
D. User notifications about new photos are lost but photos appear in feed

Solution

  1. Step 1: Understand Kafka's role in event streaming

    Kafka streams photo upload events to downstream processors like metadata extraction and feed generation.

  2. Step 2: Trace failure impact

    If Kafka is down, photo uploads still succeed because upload service writes to storage directly. Feed generation depends on Kafka events for stories update; without Kafka, stories updates stop, but feed generation for photos continues from DB. Metadata extraction is delayed but not necessarily indefinite. Notifications are handled separately.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Kafka outage stops stories update but feed photo generation continues [check]
Hint: Kafka outage blocks stories update, feed photos still served [check]
Common Mistakes:
5. At 10x scale, which component in a WebRTC video calling system is most likely to become the bottleneck first, and why?
hard
A. Media Servers, because transcoding CPU load grows linearly with concurrent calls
B. Signaling Servers, because they maintain stateful WebSocket connections and cannot easily shard sessions
C. STUN/TURN Servers, because NAT traversal requests increase exponentially with users
D. Client devices, because battery and CPU limits prevent scaling user concurrency

Solution

  1. Step 1: Analyze component scalability at 10x load

    Media Servers scale with CPU but can be horizontally scaled; STUN/TURN scale well with stateless UDP; clients are outside system control.

  2. Step 2: Identify Signaling Server bottleneck

    Signaling Servers hold stateful WebSocket connections per user, making sharding and scaling challenging.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Stateful WebSocket connections limit Signaling Server horizontal scaling [check]
Hint: Stateful WebSocket limits Signaling Server scaling first [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard