Bird
Raised Fist0

How to Structure Any System Design Answer in 45 Minutes

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
How to Structure Any System Design Answer in 45 Minutes
45 min · senior
senior

You are interviewing for a senior SDE role at a 50M DAU startup building a high-scale distributed system. The interviewer expects a clear, scalable, and trade-off aware system design answer within 45 minutes. The system must handle encryption at rest and in transit, support offline delivery, and meet a P99 latency under 200ms for core user interactions.

Constraints
  • End-to-end encryption required for user data
  • Offline delivery support with eventual consistency
  • P99 latency under 200ms for read/write operations
  • System must scale to 50 million daily active users
  • Data consistency trade-offs must be explicit
📋
Guided Steps
1/5
Step 1
Define System Boundaries and Scale
How do you define the system scope and estimate scale requirements to guide your design?
💡 Hint
Start with DAU, requests per user, and peak concurrency to derive QPS and data volume.
Key Decision
Calculate QPS as 50M DAU × 5 requests/user/day ÷ 86400 ≈ 2900 req/sec peak, guiding stateless microservices with autoscaling on Kubernetes.
Interviewer Expectation
Mid: Estimates rough QPS and mentions autoscaling. Senior: Derives exact QPS with arithmetic, explains peak vs average load, and justifies stateless microservices for elasticity.
⚠️Common Wrong Answer
Assuming uniform load without peak estimation - leads to underprovisioning and latency spikes.
Step 2
Choose Data Storage and Partitioning Strategy
What database and partitioning scheme do you select to meet scale, encryption, and latency constraints?
💡 Hint
Consider Cassandra for write-heavy workloads with partition keys supporting range queries and encryption at rest.
Key Decision
Use Apache Cassandra with partition_key=user_id for linear scalability and range queries; enable Transparent Data Encryption (TDE) for encryption at rest.
Interviewer Expectation
Mid: Names Cassandra for scale. Senior: Explains partition_key choice for data locality, why user_id over timestamp, and how TDE meets encryption requirements.
⚠️Common Wrong Answer
Using relational DB with vertical scaling - fails at 50M DAU and causes high write latency.
Step 3
Design Messaging and Offline Delivery
How do you design message delivery to support offline users and ensure eventual consistency?
💡 Hint
Use Kafka with partition key aligned to conversation or user to preserve order and enable replay for offline sync.
Key Decision
Use Apache Kafka with conversation_id as partition key to guarantee ordered delivery; offline clients consume from Kafka offsets to sync missed messages.
Interviewer Expectation
Mid: Mentions Kafka for messaging. Senior: Explains partition key choice to preserve order, offset management for offline sync, and trade-offs in consistency.
⚠️Common Wrong Answer
HTTP polling for offline sync - causes massive reconnect storms and high latency.
Step 4
Ensure Low Latency and Encryption in Transit
How do you architect the system to meet P99 < 200ms latency and enforce encryption in transit?
💡 Hint
Use TLS termination at edge load balancers and CDN caching; colocate services in same region to reduce network hops.
Key Decision
Terminate TLS at AWS ALB with TLS 1.3, colocate microservices in same AZ, and use Envoy sidecars for mutual TLS between services to maintain encryption and low latency.
Interviewer Expectation
Mid: Mentions TLS for security. Senior: Details TLS 1.3 benefits, sidecar proxies for service mesh encryption, and network topology to minimize latency.
⚠️Common Wrong Answer
Ignoring encryption in transit or using outdated TLS versions - exposes data and increases latency.
Step 5
Define Core API Endpoints
Sketch 3 core API endpoints that cover sending data, syncing offline data, and acknowledging receipt.
💡 Hint
POST /messages to send new dataGET /sync?cursor= to fetch updates since last syncPOST /ack to confirm message processing
Key Decision
Design REST endpoints: POST /messages with encrypted payload, GET /sync with cursor-based pagination for offline sync, POST /ack to update delivery status in Cassandra.
Interviewer Expectation
Mid: Lists endpoints with basic CRUD. Senior: Explains cursor-based sync for efficient offline delivery, encryption payload handling, and idempotent ACK semantics.
⚠️Common Wrong Answer
Using simple polling endpoints without cursor or ACK - leads to inefficient sync and inconsistent state.
🔌
API Design Challenge
Sketch 3 core API endpoints for your system design answer that demonstrate client interaction, data sync, and acknowledgement.
Hints
  • How does the client send data to the system?
  • How does the client synchronize state after offline periods?
  • How does the recipient acknowledge receipt or processing?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Accurate scale estimation with arithmetic and peak load consideration
  • Use of Cassandra with appropriate partition key for scale and encryption
  • Kafka partitioning strategy for ordered messaging and offline delivery
  • Encryption in transit with TLS 1.3 and service mesh mutual TLS
  • Cursor-based API design for offline sync and acknowledgement
Good to Have - 30%
  • Autoscaling microservices on Kubernetes
  • Use of Envoy sidecars for security and observability
  • Trade-off discussion on consistency vs availability
  • Latency optimization by service colocation
Bonus - 10%
  • Detailed failure mode analysis for offline sync
  • Discussion of encryption key management
  • Consideration of multi-region deployment for disaster recovery
Anti-patterns to Avoid
Ignoring peak load leading to underprovisioned services and latency spikes causing degraded user experience
Using relational databases without sharding causing write bottlenecks and system slowdowns
HTTP polling at 50M DAU: 50M reconnects/min just to check for messages - collapses under load
Terminating TLS only at edge and not between services exposing data in transit to interception
Designing APIs without cursor or acknowledgement causing inconsistent client state and data loss
Overloading single Kafka partition causing hot spots and throughput degradation
✅
Model Answer
▼ Reveal

The system design answer structures around scale estimation, data partitioning with Cassandra, ordered messaging with Kafka, encryption in transit with TLS 1.3 and Envoy, and cursor-based APIs for offline sync and acknowledgement.

Key Decisions
  • Derived QPS as 50M DAU × 5 req/user/day ÷ 86400 ≈ 2900 req/sec peak to size microservices and autoscaling on Kubernetes
  • Selected Apache Cassandra with partition_key=user_id for linear scalability and enabling Transparent Data Encryption (TDE) for encryption at rest
  • Used Apache Kafka with conversation_id as partition key to guarantee ordered message delivery and enable offline replay via offset management
  • Terminated TLS 1.3 at AWS Application Load Balancer and used Envoy sidecar proxies for mutual TLS between microservices to ensure encryption in transit and low latency
  • Designed REST APIs with POST /messages accepting encrypted payloads, GET /sync using cursor-based pagination for efficient offline data synchronization, and POST /ack for idempotent acknowledgement updating delivery status in Cassandra
  • Kubernetes for stateless microservices with autoscaling based on derived QPS
  • Apache Cassandra with user_id partition key for scalable, encrypted data storage
  • Apache Kafka with conversation_id partition key for ordered messaging and offline replay
  • TLS 1.3 termination at AWS ALB and Envoy sidecars for end-to-end encryption in transit
  • REST API with POST /messages, GET /sync with cursor, and POST /ack endpoints
  • Cursor-based pagination for efficient offline data synchronization
  • Transparent Data Encryption (TDE) enabled on Cassandra for encryption at rest
  • Use of Envoy service mesh for mutual TLS and observability

Practice

(1/5)
1. A distributed system needs to generate unique 64-bit IDs that are sortable by creation time and avoid collisions across multiple data centers without central coordination. Which component design best fits this requirement?
easy
A. Use a UUID v4 generator that produces random 128-bit IDs without ordering guarantees.
B. Use a hash of the current timestamp and server IP address as the unique ID.
C. Use a Snowflake-like ID generator combining timestamp, machine ID, and sequence number.
D. Use a centralized database auto-increment ID generator with a global lock.

Solution

  1. Step 1: Identify uniqueness and ordering requirements

    The system requires unique 64-bit IDs sortable by creation time and no central coordination to avoid bottlenecks.

  2. Step 2: Evaluate options against constraints

    UUID v4 (A) is random and not sortable; centralized DB (C) creates a bottleneck; hashing timestamp and IP (D) risks collisions and no strict ordering. Snowflake (B) combines timestamp, machine ID, and sequence to ensure uniqueness and time ordering without central coordination.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Snowflake IDs are time-sortable and unique across machines [check]
Hint: Timestamp + machine ID + sequence ensures unique sortable IDs [OK]
Common Mistakes:
2. In a system where a load balancer forwards requests to backend servers, at step 4 the backend server checks its health status before processing the request. What happens if the health check fails at this step?
easy
A. The backend server rejects the request and signals the load balancer to reroute
B. The backend server returns an error response directly to the client
C. The load balancer retries routing the request to a different healthy backend
D. The backend server processes the request anyway, risking errors

Solution

  1. Step 1: Trace backend server health check at Step 4

    If the backend server health check fails, it should not process the request (The backend server processes the request anyway, risking errors is incorrect). Returning an error directly to the client (The backend server returns an error response directly to the client) bypasses the load balancer's retry logic. The load balancer retrying without backend signaling (The load balancer retries routing the request to a different healthy backend) is incomplete.

  2. Step 2: Understand backend rejection and signaling

    The backend server rejects the request and signals the load balancer to reroute traffic to a healthy backend (The backend server rejects the request and signals the load balancer to reroute), ensuring availability.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Backend signals load balancer to reroute on health check fail [check]
Hint: Backend signals reroute on health failure [check]
Common Mistakes:
3. A system must handle 1 million writes per second and fan-out to millions of users asynchronously. Which component best handles decoupling write load from delivery?
easy
A. Use direct HTTP calls from sender to all receivers to guarantee delivery order
B. Use a relational database with sharded writes to handle all writes and fan-out synchronously
C. Use an in-memory cache like Redis to store all messages and fan-out immediately
D. Use a distributed message queue like Kafka to buffer writes and enable asynchronous fan-out

Solution

  1. Step 1: Identify the need for decoupling writes from fan-out

    Handling 1 million writes per second with synchronous fan-out is not feasible on a single machine or direct calls.

  2. Step 2: Evaluate components for asynchronous buffering

    Kafka provides durable, partitioned logs that buffer writes and allow scalable asynchronous delivery. Relational DBs can't handle this write load synchronously. Redis lacks durability guarantees for this scale. Direct HTTP calls block and don't scale.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Kafka buffers writes asynchronously, enabling high write throughput [check]
Hint: Use durable queues to decouple high write load [check]
Common Mistakes:
4. Without the automated rollback system, what exactly breaks when a misconfiguration causes a route leak?
medium
A. The system silently drops all BGP updates, causing network partition
B. Route leak detection alerts flood the monitoring system but traffic continues normally
C. The system automatically quarantines all routes, causing a complete service outage
D. Alice's traffic is routed incorrectly; operators must manually detect and fix the leak, causing prolonged outages

Solution

  1. Step 1: Trace failure without rollback

    Misconfigurations propagate incorrect routes; without rollback, manual detection delays fix.

  2. Step 2: Eliminate other failure modes

    Silent drops (A) don't occur; automatic quarantine (C) would cause total outage, not typical; alert floods (D) don't prevent traffic misrouting.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Manual fix delays cause prolonged incorrect routing [check]
Hint: No rollback means manual fix delays outages [check]
Common Mistakes:
5. At 10x scale, which component in the faceted search system is most likely to fail first and why?
hard
A. The Query Coordinator, because it cannot merge partial results fast enough under high concurrency
B. The indexing pipeline, because it cannot keep up with the volume of document updates
C. The shard-level cache, because memory pressure causes frequent evictions and cache misses spike
D. The shard nodes, because inverted index lookups become CPU-bound and cannot handle query volume

Solution

  1. Step 1: Analyze bottlenecks at 10x query scale

    Shard nodes perform CPU-intensive inverted index lookups; query volume increase causes CPU saturation first.

  2. Step 2: Compare other components

    Query Coordinator merges are parallelizable; cache evictions increase but cache can be scaled; indexing pipeline is async and can be scaled horizontally.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Shard nodes CPU-bound -> first failure point at scale [check]
Hint: Shard nodes CPU-bound first at 10x scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard