Search and recommendation in HLD - Scalability & System Analysis

Scalability Analysis - Search and recommendation

Growth Table: Search and Recommendation System

Scale	Users	Search Queries/Second	Recommendation Requests/Second	Data Size (Items, User Profiles)	System Changes
Small	100	10	5	1K items, 100 profiles	Single server, simple DB, no caching
Medium	10K	1K	500	100K items, 10K profiles	Load balancer, DB replicas, caching layer
Large	1M	100K	50K	10M items, 1M profiles	Distributed search cluster, sharded DB, ML model serving
Very Large	100M	10M	5M	1B+ items, 100M profiles	Multi-region deployment, CDN, advanced sharding, real-time streaming

First Bottleneck

At small scale, the database is the first bottleneck because it handles all search queries and recommendation data lookups. As traffic grows, the single DB cannot handle the query load and latency increases.

Scaling Solutions

• Database scaling: Add read replicas to distribute query load and use connection pooling.
• Caching: Use in-memory caches (e.g., Redis) for frequent queries and recommendation results.
• Search scaling: Deploy distributed search engines (e.g., Elasticsearch) to handle large data and queries.
• Sharding: Partition user profiles and item data across multiple databases to reduce single DB load.
• Horizontal scaling: Add more application servers behind load balancers to handle increased traffic.
• CDN: Use content delivery networks to cache static recommendation content closer to users.
• ML model serving: Use dedicated servers or services for recommendation model inference to offload app servers.

Back-of-Envelope Cost Analysis

At 1M users with 100K search queries/sec and 50K recommendation requests/sec:

• Database: Needs to handle ~150K QPS (queries per second). A single PostgreSQL instance handles ~10K QPS, so at least 15 replicas or sharded DBs are needed.
• Cache: Redis can handle ~100K ops/sec per instance, so multiple Redis nodes are required for caching.
• Network bandwidth: Assuming 1KB per query/response, total bandwidth ~150MB/s, requiring multiple 1Gbps network links or 10Gbps links.
• Storage: 10M items and 1M user profiles may require terabytes of storage, preferably on distributed storage systems.

Interview Tip

Start by clarifying scale and traffic patterns. Identify the main components: search engine, recommendation engine, database, cache, and network. Discuss bottlenecks at each scale and propose targeted solutions like caching, sharding, and horizontal scaling. Use real numbers to justify your choices and show understanding of trade-offs.

Self Check

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first and why?

Answer: Add read replicas and implement caching to reduce load on the primary database. This distributes query load and improves response times before considering more complex solutions like sharding.

Key Result

The database is the first bottleneck as traffic grows; scaling requires read replicas, caching, distributed search, and sharding to handle millions of users and queries efficiently.

Practice

1. What is the primary purpose of a search system in a large-scale application?

easy

A. To help users quickly find relevant content from a large dataset

B. To store user passwords securely

C. To manage user account settings

D. To display advertisements randomly

Solution

1. Step 1: Understand the role of search systems

   Search systems are designed to help users find information efficiently from large amounts of data.

2. Step 2: Match the purpose with options

   Only To help users quickly find relevant content from a large dataset describes helping users find relevant content quickly, which is the core function of search.

3. Final Answer:

   To help users quickly find relevant content from a large dataset -> Option A

4. Quick Check:

   Search system purpose = find relevant content [OK]

Hint: Search systems focus on finding relevant data fast [OK]

Common Mistakes:

• Confusing search with unrelated features like password storage
• Thinking search manages user settings
• Assuming search is for random content display

2. Which component is essential in a recommendation system to personalize suggestions?

easy

A. Database backup scripts

B. User behavior tracking

C. Static HTML pages

D. Load balancer configuration

Solution

1. Step 1: Identify personalization needs

   Recommendation systems personalize suggestions based on user data and behavior.

2. Step 2: Match components to personalization

   User behavior tracking collects data needed to tailor recommendations, unlike static pages or infrastructure tasks.

3. Final Answer:

   User behavior tracking -> Option B

4. Quick Check:

   Personalization needs user data = User behavior tracking [OK]

Hint: Personalization needs user data collection [OK]

Common Mistakes:

• Confusing infrastructure tasks with personalization
• Thinking static pages can personalize content
• Ignoring the role of user data

3. Consider a search system that indexes 1 million documents. If the system uses an inverted index, what is the main advantage?

medium

A. Automatically deleting old documents

B. Storing documents in a single large file

C. Encrypting all documents for security

D. Faster search queries by mapping words to document lists

Solution

1. Step 1: Understand inverted index concept

   An inverted index maps each word to the list of documents containing it, enabling quick lookups.

2. Step 2: Identify the advantage for search speed

   This mapping allows the system to find relevant documents quickly without scanning all documents.

3. Final Answer:

   Faster search queries by mapping words to document lists -> Option D

4. Quick Check:

   Inverted index = fast word-to-doc lookup [OK]

Hint: Inverted index speeds up word-based search [OK]

Common Mistakes:

• Confusing indexing with storage format
• Thinking encryption is the main index benefit
• Assuming index deletes documents automatically

4. A recommendation system is returning irrelevant suggestions. Which issue is most likely causing this?

medium

A. Using HTTPS instead of HTTP

B. Too many servers in the cluster

C. Incorrect or missing user behavior data

D. Database backup frequency is too high

Solution

1. Step 1: Analyze cause of irrelevant recommendations

   Recommendations depend on accurate user data; missing or wrong data leads to poor suggestions.

2. Step 2: Evaluate other options

   Server count, protocol choice, or backup frequency do not directly affect recommendation relevance.

3. Final Answer:

   Incorrect or missing user behavior data -> Option C

4. Quick Check:

   Bad recommendations = bad user data [OK]

Hint: Check user data quality for recommendation issues [OK]

Common Mistakes:

• Blaming infrastructure instead of data quality
• Confusing network protocols with recommendation logic
• Ignoring data collection importance

5. You are designing a scalable recommendation system for millions of users. Which approach best balances personalization and system performance?

hard

A. Use a hybrid model combining collaborative filtering and content-based filtering with offline batch processing and online updates

B. Only recommend the most popular items to all users without personalization

C. Run real-time deep learning models for every user request without caching

D. Store all user data in a single database server for simplicity

Solution

1. Step 1: Understand scalability and personalization needs

   Millions of users require efficient processing; personalization improves user experience.

2. Step 2: Evaluate approaches

   Hybrid models combine strengths of different methods. Offline batch processing reduces load, while online updates keep recommendations fresh.

3. Step 3: Reject less scalable or less personalized options

   Popular-only recommendations lack personalization. Real-time deep learning per request is costly. Single DB server is a bottleneck.

4. Final Answer:

   Use a hybrid model combining collaborative filtering and content-based filtering with offline batch processing and online updates -> Option A

5. Quick Check:

   Hybrid + batch + online = scalable personalized system [OK]

Hint: Combine offline and online methods for scalable personalization [OK]

Common Mistakes:

• Ignoring scalability by doing all processing online
• Sacrificing personalization for simplicity
• Using single server for massive data