Bird
Raised Fist0

Back-of-the-Envelope Estimation with Real Numbers - Servers, Storage, Bandwidth

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Back-of-the-Envelope Estimation with Real Numbers - Servers, Storage, Bandwidth
45 min · senior
senior

You are designing a system for a 50 million daily active user (DAU) startup that needs to provide quick, accurate back-of-the-envelope estimations for infrastructure requirements such as servers, storage, and bandwidth. The system must handle high concurrency, provide results with P99 latency under 200ms, and ensure all data is encrypted at rest and in transit. The estimations should be based on real usage patterns and historical data, supporting both online and offline delivery modes.

Constraints
  • Support 50 million DAU with peak concurrency of 5 million concurrent estimation requests
  • P99 latency for estimation results must be under 200ms
  • All data must be encrypted at rest and in transit (TLS + AES-256)
  • Support offline delivery of estimation reports via email or batch export
  • System must handle burst traffic spikes up to 2x baseline load without degradation
📋
Guided Steps
1/5
Step 1
Estimating Request Load and Throughput
How would you estimate the request volume and throughput requirements for the system based on 50M DAU and concurrency constraints?
💡 Hint
Calculate peak QPS from DAU and concurrency assumptions, consider burst factors.
Key Decision
Calculate peak QPS as 5M concurrent requests with average request duration of 200ms → 5M / 0.2s = 25M QPS peak; provision Kafka with 30 partitions to handle 25M QPS with horizontal scaling.
Interviewer Expectation
Mid: Provides rough QPS estimate from DAU and concurrency. Senior: Derives QPS with request duration, explains Kafka partitioning strategy to handle throughput and ordering.
⚠️Common Wrong Answer
Assuming DAU equals QPS directly, leading to under-provisioning and system overload during peak.
Step 2
Choosing Data Storage for Historical Usage Patterns
Which storage technology would you select to store and query historical usage data for real-time estimation calculations?
💡 Hint
Consider write-heavy workloads with time-series data and fast range queries.
Key Decision
Use Apache Cassandra with time-series data model and partition key on user_id + date to enable efficient writes and range scans for recent data.
Interviewer Expectation
Mid: Names Cassandra or similar NoSQL. Senior: Explains partition key choice for write scalability and query efficiency, rejects RDBMS due to write bottlenecks.
⚠️Common Wrong Answer
Using relational databases like MySQL for time-series data causing write bottlenecks and slow queries at scale.
Step 3
Designing the Real-Time Estimation Computation Layer
How would you architect the computation layer to deliver P99 latency under 200ms for estimation results?
💡 Hint
Consider in-memory caching, pre-aggregation, and asynchronous processing.
Key Decision
Use Redis as an in-memory cache for recent estimation results with TTL of 5 minutes, backed by a stateless microservice layer in Kubernetes for computation, and Kafka streams for asynchronous updates.
Interviewer Expectation
Mid: Suggests caching to reduce latency. Senior: Explains Redis TTL caching strategy, stateless compute pods for horizontal scaling, and Kafka streams for event-driven updates.
⚠️Common Wrong Answer
Computing estimations synchronously on every request causing high latency and poor P99 performance.
Step 4
Ensuring Data Security and Encryption
How would you ensure all data is encrypted at rest and in transit throughout the system?
💡 Hint
Consider TLS for network encryption and AES-256 for storage encryption with key management.
Key Decision
Enforce TLS 1.3 for all client-server and inter-service communication; use AWS KMS-managed AES-256 encryption for Cassandra and S3 storage; enable encryption at rest on Redis and Kafka clusters.
Interviewer Expectation
Mid: Mentions TLS and encryption at rest. Senior: Details TLS 1.3, key rotation policies, KMS integration, and encryption configuration on each component.
⚠️Common Wrong Answer
Relying only on network-level encryption without encrypting data at rest, risking data exposure on disk.
Step 5
Supporting Offline Delivery and Report Export
How would you design the offline delivery mechanism for estimation reports with acknowledgement?
💡 Hint
Consider batch processing, email delivery, and reliable message queues.
Key Decision
Use Apache Airflow for scheduled batch report generation, store reports in Amazon S3 with server-side encryption, send emails via Amazon SES with delivery tracking, and use Kafka for reliable event-driven acknowledgement processing.
Interviewer Expectation
Mid: Suggests batch jobs and email delivery. Senior: Explains Airflow DAGs for scheduling, S3 encryption, SES integration, and Kafka for reliable acknowledgement handling.
⚠️Common Wrong Answer
Implementing ad-hoc scripts without reliable retry or acknowledgement, causing report loss or duplication.
🔌
API Design Challenge
Sketch 3 core API endpoints for this estimation system that cover request submission, result retrieval, and offline report delivery.
Hints
  • How does the client submit estimation parameters and trigger computation?
  • How does the client retrieve estimation results efficiently with low latency?
  • How does the system support offline delivery and acknowledgement of report receipt?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Accurate throughput estimation with arithmetic derivation
  • Use of Kafka with partitioning strategy for high throughput and ordering
  • Selection of Cassandra with appropriate partition key for time-series data
  • Caching layer with Redis to meet latency constraints
  • Encryption at rest and in transit with specific technologies and key management
Good to Have - 30%
  • Batch processing orchestration with Apache Airflow
  • Secure report storage using encrypted S3 buckets
  • Reliable email delivery with Amazon SES and event-driven acknowledgement
  • Stateless microservices architecture for scalability and fault tolerance
Bonus - 10%
  • Detailed explanation of failure modes under burst traffic and mitigation
  • Discussion of key rotation and compliance considerations
  • Consideration of offline delivery retry and deduplication mechanisms
Anti-patterns to Avoid
Assuming DAU equals QPS causing severe under-provisioning and system crashes due to overload during peak traffic.
Using relational databases for write-heavy time-series data causing latency spikes and write bottlenecks at scale.
Synchronous computation on every request leading to P99 latency violations and poor user experience.
Encrypting only in transit but not at rest risking data breaches from disk exposure or insider threats.
Ad-hoc offline delivery without reliable retry causing data loss or duplicate report generation.
Ignoring burst traffic spikes leading to cascading failures and system downtime.
✅
Model Answer
▼ Reveal

This system design addresses the challenge of providing fast, accurate infrastructure estimations for a 50M DAU startup with strict latency and security requirements. It uses Kafka for high-throughput request ingestion, Cassandra for scalable time-series data storage, Redis caching for low-latency result retrieval, and stateless microservices for computation. Security is ensured via TLS 1.3 and AES-256 encryption managed by AWS KMS. Offline report delivery is orchestrated with Apache Airflow, storing encrypted reports in S3 and sending emails via SES with reliable acknowledgement processing through Kafka.

Key Decisions
  • Provision Kafka with 30 partitions to handle 25M QPS peak throughput and maintain ordering guarantees.
  • Use Apache Cassandra with a partition key combining user_id and date to optimize write scalability and range queries for time-series data.
  • Implement Redis caching with a 5-minute TTL to serve recent estimation results and meet P99 latency under 200ms.
  • Deploy stateless microservices on Kubernetes to enable horizontal scaling and fault tolerance for computation workloads.
  • Enforce TLS 1.3 for all network communication and use AWS KMS-managed AES-256 encryption for data at rest in Cassandra, S3, Redis, and Kafka.
  • Use Apache Airflow to schedule batch report generation workflows, storing encrypted reports in S3.
  • Send offline reports via Amazon SES with delivery tracking and use Kafka for reliable event-driven acknowledgement processing.
  • Kafka with 30 partitions to handle 25M QPS peak throughput and maintain ordering
  • Apache Cassandra with user_id + date partition key for scalable time-series data writes and queries
  • Redis with TTL caching for sub-200ms P99 latency on recent estimation results
  • Stateless microservices deployed on Kubernetes for horizontal scaling and fault tolerance
  • TLS 1.3 enforced for all network communication and AWS KMS-managed AES-256 encryption at rest
  • Apache Airflow for orchestrating batch report generation and export workflows
  • Amazon S3 with server-side encryption for secure report storage
  • Amazon SES integrated with Kafka for reliable email delivery and acknowledgement processing

Practice

(1/5)
1. A chat system must handle 1 million concurrent user requests, each requiring low-latency token streaming from a large language model. Which component is best suited to buffer and stream tokens efficiently to clients?
easy
A. A relational database with row-level locking to ensure token order
B. A batch processing system that aggregates tokens before sending
C. An in-memory message queue with backpressure support like Redis Streams
D. A distributed file system to store token chunks for streaming

Solution

  1. Step 1: Identify token streaming requirements

    Token streaming demands low latency and ordered delivery with backpressure to avoid overwhelming clients.

  2. Step 2: Evaluate component suitability

    Relational DBs (A) add latency and locking overhead; distributed file systems (C) are too slow for real-time streaming; batch processing (D) adds unacceptable delay. Redis Streams (B) supports in-memory buffering with backpressure and ordered delivery.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Redis Streams supports ordered, low-latency token streaming with backpressure [check]
Hint: In-memory queues handle streaming better than DBs or batch [check]
Common Mistakes:
2. A distributed counter system must handle millions of increments per second with minimal coordination and tolerate network partitions. Which component best supports this requirement?
easy
A. A centralized relational database with strong consistency guarantees
B. A message queue that batches increments before applying them synchronously
C. A distributed lock service to serialize all increments
D. A CRDT-based sharded counter replicated asynchronously across nodes

Solution

  1. Step 1: Identify the need for high write throughput and partition tolerance

    A centralized DB (A) and distributed lock (C) impose coordination and reduce availability. Message queue batching (D) adds latency and synchronous bottlenecks.

  2. Step 2: Recognize CRDT sharded counters allow concurrent increments without coordination and eventual consistency

    CRDTs replicate asynchronously and merge without conflicts, fitting the scale and partition tolerance needs.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    CRDTs enable concurrent, partition-tolerant increments [check]
Hint: CRDTs merge concurrent updates without coordination [check]
Common Mistakes:
3. In the photo feed generation flow, at step 4 the Feed Worker fetches stories from the Stories Service cache. What happens if the cache TTL for a user's stories has expired at this step?
easy
A. The Feed Worker asynchronously refreshes the cache but returns an empty stories section immediately
B. The Feed Worker triggers a synchronous fetch from the Stories DB and blocks until fresh stories are retrieved
C. The Feed Worker returns stale stories from a backup cache without fetching fresh data
D. The Feed Worker skips stories for that user and proceeds with only photo posts in the feed

Solution

  1. Step 1: Trace Feed Worker behavior at cache miss

    When TTL expires, the cache is stale. The Feed Worker must avoid blocking feed generation.

  2. Step 2: Understand asynchronous cache refresh

    To maintain low latency, the Feed Worker returns empty stories immediately and triggers async cache refresh. Blocking fetch (B) would increase latency. Returning stale data (A) risks outdated content. Skipping stories silently (C) loses user experience.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Async cache refresh avoids blocking feed generation [check]
Hint: Async cache refresh avoids blocking feed generation [OK]
Common Mistakes:
4. For storing aggregated logs with fast query response and cost efficiency, which storage architecture is best when logs are mostly write-once and rarely updated?
medium
A. Use a distributed relational database with strong consistency
B. Use a NoSQL key-value store with frequent compactions
C. Use a columnar data warehouse optimized for append-only writes
D. Use a distributed file system with manual indexing

Solution

  1. Step 1: Analyze log storage requirements

    Logs are write-once, large volume, and require fast analytical queries.

  2. Step 2: Evaluate columnar data warehouse

    Columnar stores optimize for append-only writes and fast scans, reducing cost and improving query speed.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Columnar warehouse -> optimized for append-only and analytics [check]
Hint: Columnar storage excels at append-only analytics [check]
Common Mistakes:
5. If the Layer 7 load balancer's HTTP header inspection module fails, what is the most likely failure mode for incoming requests?
medium
A. Requests are forwarded blindly to a default backend without routing based on URL
B. The load balancer drops all TCP connections silently without response
C. The load balancer falls back to Layer 4 forwarding, preserving source IP but ignoring HTTP routing
D. The load balancer encrypts requests incorrectly causing backend decryption errors

Solution

  1. Step 1: Understand failure impact on Layer 7 inspection

    If HTTP header inspection fails, the load balancer may fallback to Layer 4 forwarding to maintain availability.

  2. Step 2: Eliminate options inconsistent with graceful degradation

    Dropping all connections (B) is unlikely; forwarding blindly to default backend (A) loses load distribution; encryption errors (D) unrelated.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Layer 7 failure -> fallback to Layer 4 forwarding without HTTP routing [check]
Hint: Layer 7 fail -> fallback to Layer 4 forwarding [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard