Bird
Raised Fist0

Amazon - Product Catalog, Search, Cart, Order Management Pipeline

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Amazon - Product Catalog, Search, Cart, Order Management Pipeline
Design a scalable, low-latency, and highly available e-commerce backend supporting 2B DAU
hardE-Commerce and PaymentsFreq: high
Prerequisites:Inverted IndexElasticsearch
⚙️
Functional Requirements
  • Store and manage a product catalog with millions of SKUs including metadata, pricing, and inventory.
  • Provide low-latency, full-text search with filters and sorting over the product catalog.
  • Support user shopping cart operations with strong consistency and session affinity.
  • Process order lifecycle: order creation, payment processing, inventory reservation, and status updates.
  • Handle product updates, price changes, and inventory adjustments in near real-time.
  • Support personalized recommendations and promotions integration (optional extension).
📊
Non-Functional Requirements
  • Support 2 billion daily active users (DAU) globally.
  • Handle peak write throughput of approximately 925,000 writes per second (product updates, cart ops, orders).
  • Ensure 99.99% system availability with multi-region failover.
  • Achieve 100ms P99 latency for search queries and cart operations.
  • Data durability with at least 30 days retention for order and cart data.
❓
Clarifying Questions to Ask
  • What is the expected read-to-write ratio for product catalog and cart operations?
  • Are product updates and inventory changes real-time or batch processed?
  • What consistency guarantees are required for cart and order operations?
  • Is the search system expected to support multi-language and geo-based filtering?
  • What is the expected size and growth rate of the product catalog?
  • Are there SLAs for order processing time from creation to fulfillment?
Out of Scope
✗ Payment gateway internals and fraud detection systems.
✗ Recommendation engine design beyond basic integration points.
✗ Warehouse management and logistics tracking.
✗ User authentication and profile management.
📌
Assumptions
  • 2 billion DAU globally.
  • Each user generates on average 40 messages (writes) per day (product views, cart ops, orders).
  • Average message size is 100 bytes (metadata, event payload).
  • Peak write traffic is 10x average during sales events.
⚡
QPS Calculation
write qps2,000,000,000 × 40 / 86,400 ≈ 925,926 writes/sec
peak write qps925,926 × 10 = 9,259,260 writes/sec
read qpsAssuming 10 reads per write, read_qps = 9,259,260 × 10 = 92,592,600 reads/sec
💾
Storage Estimate
daily storage925,926 writes/sec × 100 bytes × 86,400 sec = 8 TB/day
retention days30
total storage8 TB × 30 = 240 TB raw
with replication240 TB × 3 (replication factor) = 720 TB total
🖥️
Server Estimate
catalog dbAssuming Cassandra nodes handle 50K writes/sec each, need 185 nodes for peak writes
search clusterElasticsearch cluster sized for 100M queries/sec with 200 nodes (based on shard and replica count)
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          |  User Clients  |
          +-------+--------+
                  |
          +-------v--------+
          | API Gateway    |
          +-------+--------+
                  |
  +---------------+----------------+----------------+
  |               |                |                |
  |               |                |                |
  v               v                v                v
Product Catalog  Search          Cart Service    Order Service
  DB (Cassandra)  (Elasticsearch) (Redis + DB)    (Cassandra + Kafka)
                  |                |                |
                  +----------------+----------------+
                                   |
                            +------v-------+
                            |  Kafka Queue |
                            +------+-------+
                                   |
                            +------v-------+
                            | Order Worker |
                            +--------------+
🧩
Components
API Gateway
Entry point for all client requests, routing to appropriate backend services with authentication and rate limiting.
Why thisAWS API Gateway supports global scaling with integrated throttling and caching to handle 100M+ QPS.
Why notNginx reverse proxy lacks native global scaling and integrated rate limiting at this scale.
Product Catalog DB (Apache Cassandra)
Stores product metadata, pricing, and inventory with high write throughput and horizontal scalability.
Why thisCassandra provides linear write scalability, tunable consistency, and multi-region replication suitable for 925K+ writes/sec.
Why notPostgreSQL requires complex manual sharding and does not scale linearly for 925K writes/sec.
Search Service (Elasticsearch)
Provides full-text search with filters, sorting, and faceted navigation over product catalog.
Why thisElasticsearch supports inverted indexes, distributed search, and near real-time indexing for low-latency queries.
Why notSolr lacks the same level of cloud-native scalability and community support for near real-time indexing.
Cart Service (Redis + Cassandra)
Manages user shopping carts with low latency and strong consistency for session data and persistence.
Why thisRedis supports sub-millisecond lookups with TTL for session expiry; Cassandra persists cart state for durability.
Why notMemcached lacks persistence and TTL management, risking data loss on failures.
Order Service (Cassandra + Kafka)
Handles order lifecycle events, inventory reservation, and status updates asynchronously.
Why thisKafka provides durable, ordered event streaming for decoupling order processing; Cassandra stores order state reliably.
Why notRabbitMQ does not scale as well for 9M+ messages/sec and lacks partitioned log semantics.
Kafka Queue
Asynchronous event bus for order processing and inventory updates to decouple services and enable scalability.
Why thisKafka supports high throughput, partitioned logs, and consumer groups for parallel processing at scale.
Why notAmazon SQS has higher latency and lower throughput unsuitable for real-time order processing.
Order Worker
Consumes Kafka events to process orders, update inventory, and trigger downstream workflows.
Why thisStateless microservices deployed on Kubernetes enable horizontal scaling and fault isolation.
Why notMonolithic order processors cannot scale horizontally and are prone to single points of failure.

Practice

(1/5)
1. In the booking flow, at step 4 the Availability Service checks the calendar cache before confirming a booking. What happens if the cache entry for the requested dates is stale or missing at this step?
easy
A. The service immediately rejects the booking request without fallback
B. The service queries the persistent calendar database synchronously to verify availability
C. The service assumes availability and proceeds to reserve the booking optimistically
D. The service delays the booking confirmation until a cache refresh completes asynchronously

Solution

  1. Step 1: Trace Availability Service behavior at step 4

    The service uses cache for fast reads but must ensure correctness before booking confirmation.

  2. Step 2: Handle stale or missing cache entries

    Rejecting immediately (A) reduces UX; optimistic booking (C) risks double bookings; async delay (D) hurts latency. Synchronous DB query (B) ensures correctness.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Fallback to DB ensures accurate availability check on cache miss [check]
Hint: Cache miss triggers synchronous DB check for correctness [check]
Common Mistakes:
2. For a globally distributed collaborative editor with many offline users, which approach better supports eventual consistency and offline edits, and why?
medium
A. Use Operational Transformation (OT) because it requires a central server to order operations
B. Use a relational database with strict locking to serialize all edits globally
C. Use CRDTs because they allow local operations to be merged without coordination and support offline edits
D. Use a message queue with timestamp ordering to ensure global operation order

Solution

  1. Step 1: Compare OT and CRDT for offline and distributed scenarios

    OT typically requires a central server to order and transform operations, making offline support complex.

  2. Step 2: Understand CRDT advantages

    CRDTs are designed for decentralized environments, allowing local operations to be merged eventually without coordination, ideal for offline edits and global distribution.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    CRDTs enable offline edits and eventual consistency without central coordination [check]
Hint: CRDTs merge offline edits without central coordination [OK]
Common Mistakes:
3. At 10x scale, which component in a Blue-Green deployment system is most likely to fail first and why?
hard
A. Deployment pipeline because build servers cannot handle 10x concurrent builds, delaying releases
B. Green environment servers because maintaining two full production environments doubles resource usage and costs
C. Load balancer because it must instantly switch traffic for millions of users, causing connection storms
D. Feature flag service because it cannot handle 10x toggle evaluations per request

Solution

  1. Step 1: Identify scaling bottlenecks

    Blue-Green requires two full environments; load balancer scales horizontally; feature flags scale independently.

  2. Step 2: Analyze deployment pipeline

    Build servers face 10x concurrent builds, causing queueing and delays, blocking releases.

  3. Step 3: Compare other components

    Load balancer and feature flags can be sharded; doubling environment cost is expensive but not a failure.

  4. Final Answer:

    Option A -> Option A

  5. Quick Check:

    Build servers bottleneck at 10x concurrent builds, delaying deployments [check]
Hint: Build servers choke first at 10x concurrent builds, blocking deploys [OK]
Common Mistakes:
4. At 10x scale, which component in a consistent hashing distributed cache system is most likely to fail first, and why?
hard
A. Consistent hashing ring metadata service, because it becomes a bottleneck updating and distributing ring changes rapidly
B. Cache nodes, because their memory capacity limits are exceeded by increased key volume
C. Client-side hashing logic, because it cannot handle the increased number of nodes efficiently
D. Network switches, because increased traffic saturates bandwidth causing packet loss

Solution

  1. Step 1: Identify scaling bottlenecks

    Cache nodes scale horizontally; memory limits can be increased. Client hashing logic is local and scales well. Network switches scale with infrastructure upgrades.

  2. Step 2: Consistent hashing ring metadata bottleneck

    The ring metadata service must update and distribute node membership changes; at 10x scale, frequent updates cause high coordination overhead and latency, becoming the first failure point.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Metadata service update overhead limits scaling first [check]
Hint: Metadata service update overhead limits scaling first [check]
Common Mistakes:
5. At 10x scale, which component in the SLO error budget tracking system is most likely to become the bottleneck first, and why?
hard
A. The metrics aggregation service, because it maintains stateful counters and cannot shard error budget calculations easily
B. The dashboard visualization layer, because rendering 10x more data overwhelms client browsers
C. The raw event ingestion pipeline, because Kafka partitions can be increased indefinitely without limit
D. The alerting system, because it can batch alerts and scale horizontally without state

Solution

  1. Step 1: Identify stateful components that limit horizontal scaling

    The metrics aggregation service maintains counters and error budget state, which are hard to shard without losing accuracy.

  2. Step 2: Evaluate other components' scalability

    Dashboards (A) are client-side and less critical; Kafka (C) scales well with partitions; alerting (D) can batch and scale horizontally.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Stateful aggregation -> sharding limits -> bottleneck at scale [check]
Hint: Stateful aggregation limits sharding, causing first bottleneck at scale [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard