Bird
Raised Fist0

Design a Search Ranking System - Feature Store, Model Serving, A/B Testing

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Design a Search Ranking System - Feature Store, Model Serving, A/B Testing
45 min · senior
senior

You are designing a search ranking system for a startup with 50 million daily active users (DAU). The system must support real-time ranking with a P99 latency under 200ms, handle encrypted user data end-to-end, support offline feature delivery for batch model training, and enable controlled A/B testing of ranking models. The system should scale to handle 10,000 queries per second at peak and support feature updates with minimal staleness.

Constraints
  • End-to-end encryption of user features and query data at rest and in transit
  • Offline feature delivery pipeline for batch model training with daily refresh
  • P99 latency for online ranking under 200ms including feature fetch and model inference
  • Support controlled A/B testing with traffic split and metric collection
  • Scale to 10K QPS peak query load with 50M DAU
  • Feature store must support both online low-latency and offline high-throughput access
📋
Guided Steps
1/5
Step 1
Designing the Feature Store Architecture
How would you design the feature store to support both low-latency online feature retrieval and high-throughput offline batch processing?
💡 Hint
Consider separate storage optimized for online and offline access patterns with a unified feature schema.
Key Decision
Use Redis with RedisJSON for online feature store to achieve sub-10ms fetch latency, and Apache Hudi on S3 for offline feature storage enabling incremental batch updates and time travel queries.
Interviewer Expectation
Mid-level: Names Redis and Hudi for online/offline. Senior: Explains RedisJSON for nested feature access and Hudi's incremental compaction for efficient offline refresh.
⚠️Common Wrong Answer
Using a single relational database like MySQL for both online and offline features, causing high latency and poor batch throughput.
Step 2
Model Serving Infrastructure
What technology and architecture would you choose for serving ranking models with P99 latency under 200ms at 10K QPS?
💡 Hint
Think about model containerization, inference caching, and horizontal scaling with autoscaling capabilities.
Key Decision
Deploy models using TensorFlow Serving with gRPC endpoints behind an Envoy proxy for load balancing, combined with Redis as an inference cache for repeated queries to reduce latency.
Interviewer Expectation
Mid-level: Mentions TensorFlow Serving and load balancing. Senior: Details Envoy for L7 routing, Redis cache TTL strategy, and autoscaling based on CPU and latency metrics.
⚠️Common Wrong Answer
Serving models directly from batch jobs or using synchronous HTTP calls without caching, leading to high latency and poor scalability.
Step 3
Implementing A/B Testing for Ranking Models
How would you implement controlled A/B testing to route traffic and collect metrics for different ranking models?
💡 Hint
Consider user bucketing, consistent hashing, and metric aggregation pipelines.
Key Decision
Use LaunchDarkly for feature flagging and traffic split with consistent hashing on user IDs, combined with Kafka for streaming metric events to a real-time analytics platform like Apache Pinot.
Interviewer Expectation
Mid-level: Names feature flags and Kafka. Senior: Explains consistent hashing to ensure stable user bucket assignment and Kafka topic partitioning for metric scalability.
⚠️Common Wrong Answer
Randomly assigning users on each request causing inconsistent experiences and noisy metrics.
Step 4
Ensuring Data Encryption and Security
How do you ensure end-to-end encryption of user and query data in the system, including at rest and in transit?
💡 Hint
Think about TLS, encryption keys management, and encryption at storage layers.
Key Decision
Enforce TLS 1.3 for all network communication, use AWS KMS for envelope encryption of features stored in Redis and S3, and enable Redis AUTH with TLS for client connections.
Interviewer Expectation
Mid-level: Mentions TLS and encryption at rest. Senior: Details envelope encryption with KMS, key rotation policies, and Redis AUTH with TLS enforcement.
⚠️Common Wrong Answer
Relying only on network-level encryption without encrypting data at rest, risking data exposure if storage is compromised.
Step 5
Handling Offline Feature Delivery and Model Training
How would you design the offline pipeline to deliver features for batch model training with daily refresh and minimal staleness?
💡 Hint
Consider incremental data ingestion, feature computation frameworks, and storage format optimized for analytics.
Key Decision
Use Apache Spark on EMR to process incremental feature updates stored in Apache Hudi on S3, enabling efficient upserts and time travel for training data snapshots.
Interviewer Expectation
Mid-level: Names Spark and S3. Senior: Explains Hudi's incremental compaction and snapshot isolation for consistent training datasets.
⚠️Common Wrong Answer
Full daily batch recomputation without incremental updates, causing long pipeline runtimes and stale features.
🔌
API Design Challenge
Sketch 3 core API endpoints for the Search Ranking System: feature retrieval, model inference request, and A/B test traffic assignment.
Hints
  • How does the client request features for a query and user context?
  • How does the client submit a ranking request and receive scored results?
  • How does the system assign users or queries to A/B test buckets and report metrics?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Separate online and offline feature stores with appropriate technologies
  • Low-latency model serving architecture with caching and autoscaling
  • Controlled A/B testing with stable user bucketing and metric collection
  • End-to-end encryption including at rest and in transit
Good to Have - 30%
  • Incremental offline feature delivery pipeline with snapshot isolation
  • Use of feature flagging platform for traffic control
  • Real-time analytics platform for A/B test metrics
  • Autoscaling and load balancing for model serving
Bonus - 10%
  • Detailed API design for feature retrieval, model inference, and A/B test assignment
  • Discussion of key rotation and security best practices
  • Failure mode analysis for cache misses and stale features
Anti-patterns to Avoid
Using a single relational database for both online and offline features causes high latency and throughput bottlenecks, leading to poor user experience and slow batch processing.
Serving models synchronously without caching leads to high P99 latency and poor scalability, causing system overload under peak traffic.
Random user assignment in A/B testing causes inconsistent user experience and noisy metrics, making it difficult to draw reliable conclusions.
Relying solely on network encryption without encrypting data at rest risks data exposure if storage systems are compromised, violating compliance and security requirements.
✅
Model Answer
▼ Reveal

This design addresses the need for a scalable, low-latency search ranking system supporting encrypted data, offline feature delivery, and controlled A/B testing. It uses specialized technologies for online and offline feature stores, a robust model serving infrastructure with caching and autoscaling, and a stable A/B testing framework with consistent user bucketing and real-time metric collection. Security is ensured via TLS and envelope encryption with key management.

Key Decisions
  • Separate online and offline feature stores to optimize for latency and throughput respectively
  • Use RedisJSON to efficiently access nested features in online store
  • Leverage Apache Hudi's incremental compaction and snapshot isolation for offline feature freshness
  • Deploy TensorFlow Serving behind Envoy for load balancing and L7 routing, with Redis caching to reduce inference latency
  • Implement consistent hashing on user IDs for stable A/B test bucket assignment to ensure consistent user experience
  • Stream metrics via Kafka to Apache Pinot for scalable real-time analytics
  • Enforce TLS 1.3 for all network communication and use AWS KMS for envelope encryption with key rotation policies
  • Use Apache Spark on EMR for incremental offline feature computation to minimize staleness
  • Redis with RedisJSON for online feature store to achieve sub-10ms feature fetch latency
  • Apache Hudi on S3 for offline feature storage enabling incremental batch updates and time travel queries
  • TensorFlow Serving with gRPC endpoints behind Envoy proxy for scalable low-latency model inference
  • Redis as an inference cache to reduce repeated query latency
  • LaunchDarkly for feature flagging and traffic split with consistent hashing on user IDs for stable A/B testing
  • Kafka for streaming metric events to Apache Pinot for real-time analytics
  • TLS 1.3 enforced for all network communication and AWS KMS for envelope encryption of data at rest
  • Apache Spark on EMR for incremental offline feature computation with Hudi storage format

Practice

(1/5)
1. In the order matching flow, at step 5 the Matching Engine receives a new buy order for stock XYZ. What state does the Matching Engine hold immediately after processing this order if no matching sell orders exist?
easy
A. The order is marked as matched and removed from the order book
B. The order is added to the buy side of the order book and marked as open
C. The order is rejected due to lack of matching sell orders
D. The order is forwarded to the Market Data Feed before being stored

Solution

  1. Step 1: Trace Matching Engine behavior at step 5

    When no matching sell orders exist, the buy order is added to the order book's buy side and marked as open, waiting for future matches.

  2. Step 2: Eliminate other options

    The order is marked as matched and removed from the order book is wrong because no match occurred; The order is rejected due to lack of matching sell orders is wrong because orders are not rejected for no matches; The order is forwarded to the Market Data Feed before being stored is wrong because Market Data Feed only publishes after matching or order book update.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Unmatched order -> added to order book as open [check]
Hint: Unmatched orders stay open in order book [check]
Common Mistakes:
2. A web application experiences increasing CPU load on a single database server, causing query latency spikes. Which scaling approach best addresses this bottleneck without changing the application code?
easy
A. Distribute the database load by sharding data across multiple servers (horizontal scaling)
B. Add more CPU cores and RAM to the existing database server (vertical scaling)
C. Use a caching layer in front of the database to reduce read load
D. Switch from SQL to NoSQL database to improve write throughput

Solution

  1. Step 1: Identify the bottleneck as CPU and memory on a single DB server

    Vertical scaling adds resources to the existing server to handle more load without architectural changes.

  2. Step 2: Evaluate horizontal scaling and other options

    Horizontal scaling requires application changes for sharding; caching reduces reads but not CPU-bound writes; switching DB types is a different trade-off.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Vertical scaling adds resources to one server without code changes [check]
Hint: Vertical scaling adds power to one machine [check]
Common Mistakes:
3. If the event ingestion pipeline fails to persist incoming metric data, what is the exact failure observed in the alerting system?
medium
A. Threshold and anomaly detectors receive incomplete data, causing missed or false alerts
B. Alerts are generated with delayed timestamps but eventually delivered
C. Composite alerts continue to function normally using cached inputs
D. The system switches to a backup data source automatically without alert impact

Solution

  1. Step 1: Role of event ingestion pipeline

    This pipeline persists raw metric data for downstream alerting components.

  2. Step 2: Consequence of pipeline failure

    Without persistence, threshold and anomaly detectors get incomplete or missing data, leading to missed alerts or false positives.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Missing data -> detectors miss or misfire alerts [check]
Hint: No data persistence -> detectors get incomplete data, miss alerts [check]
Common Mistakes:
4. For delivering real-time location updates to mobile clients, which approach is better considering intermittent mobile network connectivity and battery constraints?
medium
A. Use WebSocket connections with automatic reconnection and heartbeat pings
B. Use persistent gRPC streaming connections to push updates continuously
C. Use HTTP long polling to simulate real-time updates
D. Use periodic client polling every few seconds to fetch updates

Solution

  1. Step 1: Evaluate persistent connection options

    gRPC streaming is efficient but less supported on mobile platforms and can drain battery.

  2. Step 2: Compare WebSocket and polling

    WebSocket supports bidirectional communication with reconnection and heartbeats, balancing real-time and battery use better than polling.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    WebSocket balances real-time updates and mobile constraints [check]
Hint: WebSocket with reconnection beats gRPC streaming on mobile [check]
Common Mistakes:
5. At 10x scale, which component in a cache stampede prevention system is most likely to fail first, and why?
hard
A. Cache servers, because they run out of memory storing more keys
B. Backend database, because it cannot handle the increased read load from cache misses
C. Distributed lock service, because lock contention and coordination overhead grow exponentially
D. Load balancers, because they cannot distribute requests evenly at high concurrency

Solution

  1. Step 1: Identify bottlenecks at 10x scale

    Backend and cache scale horizontally; load balancers scale well.

  2. Step 2: Analyze distributed lock service

    Lock service faces increased contention and coordination overhead, causing latency and failures.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Distributed lock service bottlenecks first due to coordination overhead [check]
Hint: Lock service bottlenecks first due to exponential coordination overhead [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard