Bird
Raised Fist0

Document Databases - MongoDB Data Modeling Patterns at Scale

HLDmediumInterview freq: medium
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Document Databases - MongoDB Data Modeling Patterns at Scale
45 min · senior
senior

You are designing the data model and access patterns for a 50 million daily active user (DAU) social media platform using MongoDB. The platform requires end-to-end encryption at rest and in transit, supports offline data delivery with eventual consistency, and must maintain a P99 read latency under 200ms globally. Users generate billions of documents daily, including posts, comments, and user profiles. The system must support efficient queries for user timelines, feed generation, and search by tags and metadata.

Constraints
  • End-to-end encryption at rest and in transit is mandatory
  • Offline data delivery with eventual consistency for mobile clients
  • P99 read latency must be under 200ms globally
  • Support billions of documents daily with high write throughput
  • Complex queries on nested documents and metadata must be efficient
📋
Guided Steps
1/5
Step 1
Choosing the MongoDB Data Model: Embedding vs Referencing
How would you design the document schema for posts and comments to optimize for read latency and write throughput at 50M DAU scale?
💡 Hint
Consider MongoDB document size limits and query patterns for timelines and comments.
Key Decision
Use embedding for comments within posts when comment count is low and referencing with separate MongoDB collections for high comment volume - balances read efficiency and document size limits in MongoDB.
Interviewer Expectation
Mid: Mentions embedding for nested data. Senior: Explains MongoDB 16MB document size limit and how embedding impacts update contention and query latency.
⚠️Common Wrong Answer
Embed all comments inside posts regardless of volume - leads to oversized documents and slow writes due to document-level locking.
Step 2
Indexing Strategy for Efficient Queries
What indexes would you create to support fast timeline queries and tag-based searches while maintaining write performance?
💡 Hint
Think about compound indexes, partial indexes, and index cardinality in MongoDB.
Key Decision
Create compound indexes on {user_id, created_at} fields for timeline queries and sparse indexes on tags fields - optimizes query latency under 200ms while minimizing write amplification in MongoDB.
Interviewer Expectation
Mid: Names indexes on user_id and timestamps. Senior: Explains trade-offs of compound vs single-field indexes and how sparse indexes reduce index size for optional fields.
⚠️Common Wrong Answer
Create too many indexes including on low-cardinality fields - causes write throughput degradation and increased storage overhead.
Step 3
Handling Offline Data Delivery and Sync
How would you design the data sync mechanism for mobile clients that can be offline and require eventual consistency?
💡 Hint
Consider MongoDB Change Streams and conflict resolution strategies.
Key Decision
Use MongoDB Change Streams with resume tokens for incremental sync and implement client-side vector clocks for conflict detection and resolution.
Interviewer Expectation
Mid: Suggests polling or full sync. Senior: Explains Change Streams scalability and how vector clocks enable deterministic conflict resolution.
⚠️Common Wrong Answer
Use periodic full data syncs - causes excessive bandwidth and high latency, breaking P99 SLA.
Step 4
Ensuring Encryption and Security Compliance
How do you enforce end-to-end encryption at rest and in transit while maintaining query efficiency?
💡 Hint
Think about MongoDB’s field-level encryption and TLS, and how encryption impacts indexing and querying.
Key Decision
Use MongoDB Client-Side Field Level Encryption (CSFLE) for sensitive fields combined with TLS for transport - protects data without exposing plaintext to the server and maintains index usability on non-encrypted fields.
Interviewer Expectation
Mid: Mentions TLS and encryption at rest. Senior: Details CSFLE trade-offs, indexing limitations on encrypted fields, and key management.
⚠️Common Wrong Answer
Encrypt entire documents server-side - breaks indexing and causes full collection scans, increasing latency beyond 200ms.
Step 5
Scaling MongoDB Cluster for High Write Throughput
How would you shard your MongoDB cluster to handle billions of writes daily and maintain low latency?
💡 Hint
Consider shard key selection and balancing write hotspots.
Key Decision
Shard on a compound key {user_id, created_at} to distribute writes evenly and enable efficient range queries for timelines in MongoDB.
Interviewer Expectation
Mid: Picks user_id as shard key. Senior: Explains why user_id alone causes hotspots and how adding created_at enables write distribution and range scans.
⚠️Common Wrong Answer
Shard on monotonically increasing _id - causes write hotspots on single shard, degrading throughput and latency.
🔌
API Design Challenge
Sketch 3 core API endpoints for the MongoDB-backed service: one for creating documents, one for querying user timelines, and one for syncing offline changes.
Hints
  • How does the client send new or updated documents securely?
  • How does the system efficiently query and paginate user timelines?
  • How does the client acknowledge and resolve conflicts during offline sync?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Clear explanation of embedding vs referencing trade-offs in MongoDB with document size limits
  • Shard key choice that balances write distribution and query efficiency
  • Use of MongoDB Change Streams for offline sync rather than polling or full sync
  • Implementation of Client-Side Field Level Encryption (CSFLE) for end-to-end encryption
Good to Have - 30%
  • Detailed indexing strategy including compound and sparse indexes
  • Conflict resolution strategy using vector clocks or similar
  • Consideration of TLS and key management for encryption
  • Awareness of MongoDB document-level locking impact on writes
Bonus - 10%
  • Discussion of multi-region cluster deployment for global latency
  • Use of MongoDB Atlas features for automated scaling and backups
Anti-patterns to Avoid
Embedding unbounded arrays leads to MongoDB 16MB document size limit breaches and slow writes due to document-level locking
Sharding on monotonically increasing keys causes write hotspots on a single shard, degrading throughput and increasing latency
Full data sync for offline clients causes excessive bandwidth usage and high latency, breaking P99 SLA
Encrypting entire documents server-side breaks MongoDB indexes and causes full collection scans, increasing latency beyond 200ms
Over-indexing low-cardinality fields causes write amplification and storage bloat, reducing write throughput
Polling for data changes instead of using MongoDB Change Streams causes scalability collapse due to excessive load
✅
Model Answer
▼ Reveal

Design MongoDB data models using embedding for small nested data and referencing for large or growing arrays to avoid the 16MB document size limit. Use compound indexes on {user_id, created_at} to support efficient timeline queries achieving P99 read latency under 200ms. Implement offline sync using MongoDB Change Streams with resume tokens and client-side vector clocks for conflict resolution. Enforce end-to-end encryption with MongoDB Client-Side Field Level Encryption (CSFLE) for sensitive fields and TLS for transport security. Shard the MongoDB cluster on a compound key {user_id, created_at} to evenly distribute write load and enable efficient range queries.

Key Decisions
  • Embedding comments within posts when comment count is low to optimize read latency and reduce joins
  • Referencing comments in a separate collection when comment volume is high to avoid exceeding MongoDB 16MB document size limit
  • Creating compound indexes on {user_id, created_at} to support efficient timeline queries and maintain P99 latency under 200ms
  • Using sparse indexes on optional metadata fields to reduce index size and write amplification
  • Leveraging MongoDB Change Streams with resume tokens for efficient incremental sync of offline client changes
  • Implementing client-side vector clocks to detect and resolve conflicts deterministically during offline sync
  • Applying MongoDB Client-Side Field Level Encryption (CSFLE) to protect sensitive data fields without sacrificing queryability on non-encrypted fields
  • Enforcing TLS for all client-server communication to secure data in transit
  • Sharding the MongoDB cluster on compound key {user_id, created_at} to distribute write load evenly and enable efficient range queries
  • MongoDB embedding vs referencing: embedding for small nested data, referencing for large or growing arrays to avoid 16MB limit
  • Compound indexes on {user_id, created_at} for timeline queries to achieve P99 < 200ms
  • MongoDB Change Streams with resume tokens for efficient incremental offline sync
  • MongoDB Client-Side Field Level Encryption (CSFLE) for end-to-end encryption without sacrificing queryability on non-encrypted fields
  • TLS for all client-server communication to secure data in transit
  • Sharding on compound key {user_id, created_at} to evenly distribute write load and enable range queries
  • Use of vector clocks on client for conflict detection and resolution during offline sync
  • Sparse and partial indexes on optional metadata fields to reduce index size and write amplification

Practice

(1/5)
1. A large e-commerce platform needs to shard its user data to support efficient range queries on user signup dates. Which sharding strategy best fits this requirement?
easy
A. Directory-based sharding to map each user ID to a specific shard via a lookup table
B. Hash-based sharding to evenly distribute users across shards regardless of signup date
C. Range-based sharding to group users by signup date intervals for efficient range queries
D. Random sharding to balance load without considering signup date

Solution

  1. Step 1: Identify query pattern

    Range queries on signup dates require data locality by date ranges to avoid scatter-gather queries.

  2. Step 2: Match sharding strategy

    Range-based sharding partitions data by contiguous key ranges, enabling efficient range scans.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Range sharding groups data by key intervals for range queries [check]
Hint: Range sharding groups contiguous keys for range queries [check]
Common Mistakes:
2. A distributed caching system needs to assign keys to cache nodes such that minimal remapping occurs when nodes join or leave. Which component best handles this requirement?
easy
A. Use Rendezvous Hashing to assign keys to nodes with highest hash score
B. Use a simple modulo-based hash function on node count
C. Use a centralized directory service mapping keys to nodes
D. Use consistent hashing with virtual nodes and a hash ring

Solution

  1. Step 1: Understand minimal remapping requirement

    Modulo-based hashing (Use a simple modulo-based hash function on node count) causes massive remapping when node count changes. Centralized directory (Use a centralized directory service mapping keys to nodes) adds single point of failure and latency. Consistent hashing with virtual nodes (Use consistent hashing with virtual nodes and a hash ring) reduces remapping but is more complex and less direct than Rendezvous Hashing.

  2. Step 2: Identify Rendezvous Hashing benefits

    Rendezvous Hashing assigns keys to nodes by computing scores and picking the highest, minimizing remapping and avoiding ring maintenance.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Rendezvous Hashing picks highest score node -> minimal remapping [check]
Hint: Highest hash score wins key assignment [check]
Common Mistakes:
3. In the message delivery trace, at step 4 the Delivery Worker checks the subscriber's presence TTL in Redis. What happens if the TTL has expired at this step?
easy
A. The Delivery Worker forwards the message to a backup subscriber
B. The Delivery Worker marks the message as undeliverable and stops retries
C. The Delivery Worker queues the message for later retry after refreshing TTL
D. The Delivery Worker immediately retries delivery without delay

Solution

  1. Step 1: Understand Delivery Worker behavior on subscriber presence TTL expiry

    If TTL expired, subscriber is considered offline. Delivery Worker does not retry immediately (The Delivery Worker immediately retries delivery without delay) or mark undeliverable (The Delivery Worker marks the message as undeliverable and stops retries) because retries are needed. It does not refresh TTL (The Delivery Worker queues the message for later retry after refreshing TTL) as TTL is subscriber-driven.

  2. Step 2: Recognize fallback to backup subscriber for high availability

    Delivery Worker forwards message to backup subscriber or alternative delivery path to ensure eventual delivery.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Expired TTL -> fallback delivery to backup subscriber [check]
Hint: Expired TTL triggers fallback delivery, not immediate retry [check]
Common Mistakes:
4. At 10x scale, which component in the cart service is most likely to fail first and why?
hard
A. The cache layer because it cannot evict entries fast enough causing stale cart data
B. The session management service because it maintains stateful WebSocket connections that cannot be easily sharded
C. The relational database because it cannot handle the increased write volume for cart updates
D. The load balancer because it cannot distribute traffic evenly across cart servers

Solution

  1. Step 1: Identify stateful bottleneck

    Session management holds WebSocket connections statefully, limiting horizontal scaling and resharding.

  2. Step 2: Compare other components

    Relational DB (A) can be scaled with sharding; cache eviction (B) is manageable with TTL tuning; load balancer (D) scales horizontally.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Stateful WebSocket sessions -> scaling bottleneck at 10x load [check]
Hint: Stateful WebSocket sessions block scaling first [check]
Common Mistakes:
5. At 10x scale, which component in the load balancer system is most likely to fail first and why?
hard
A. Load balancer routing tables, because stateful session persistence cannot shard easily
B. Health check workers, due to increased probe volume causing CPU exhaustion
C. Backend servers, because they cannot handle increased traffic without scaling
D. DNS resolvers, due to increased query volume causing cache misses

Solution

  1. Step 1: Identify stateful bottlenecks at scale

    Routing tables maintaining session persistence are stateful and hard to shard, causing scaling limits.

  2. Step 2: Compare other components

    Health checks scale horizontally; backend servers scale independently; DNS resolvers can be cached and distributed.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Stateful routing tables limit scaling due to session affinity [check]
Hint: Stateful routing tables limit scaling first [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard