Search and filter design in LLD - Architecture Diagram

System Overview - Search and filter design

This system allows users to search and filter large sets of data quickly and efficiently. It supports keyword searches and multiple filter options to narrow down results, ensuring fast response times even with high user traffic.

Architecture Diagram

User
  |
  v
Load Balancer
  |
  v
API Gateway
  |
  v
Search Service <--> Cache Layer
  |
  v
Search Index (Elasticsearch)
  |
  v
Primary Database

Components

User

client

Initiates search and filter requests

Load Balancer

load_balancer

Distributes incoming requests evenly to API Gateway instances

API Gateway

api_gateway

Handles authentication, routing, and request validation

Search Service

service

Processes search queries and applies filters

Cache Layer

cache

Stores recent search results to reduce latency

Search Index (Elasticsearch)

search_engine

Indexes data for fast full-text search and filtering

Primary Database

database

Stores the authoritative data source for indexing and updates

Request Flow - 11 Hops

User → Load Balancer

Load Balancer → API Gateway

API Gateway → Search Service

Search Service → Cache Layer

Cache Layer → Search Service

Search Service → Search Index (Elasticsearch)

Search Index (Elasticsearch) → Search Service

Search Service → Cache Layer

Search Service → API Gateway

API Gateway → Load Balancer

Load Balancer → User

Failure Scenario

Component Fails:Search Index (Elasticsearch)

Impact:Search queries cannot be processed, resulting in failed or delayed search responses

Mitigation:Use replicated search index clusters with failover; fallback to cached results if available; alert system operators for quick recovery

Architecture Quiz - 3 Questions

Which component is responsible for distributing incoming user requests evenly?

A API Gateway

B Load Balancer

C Search Service

D Cache Layer

Design Principle

This design uses a layered approach with caching and a dedicated search index to provide fast and scalable search and filtering. It separates concerns by using an API Gateway for routing and validation, and a cache to reduce load on the search engine, improving performance and reliability.

Practice

1. What is the main purpose of adding filters in a search system?

easy

A. To slow down the search process for accuracy

B. To increase the total number of search results

C. To narrow down search results based on user preferences

D. To remove the search bar from the interface

Solution

1. Step 1: Understand the role of filters in search

   Filters help users reduce the number of results by selecting specific criteria.

2. Step 2: Identify the effect of filters on results

   Filters narrow results to match user preferences, making search faster and more relevant.

3. Final Answer:

   To narrow down search results based on user preferences -> Option C

4. Quick Check:

   Filters narrow results = C [OK]

Hint: Filters reduce results to match user needs quickly [OK]

Common Mistakes:

• Thinking filters increase results
• Assuming filters slow down search intentionally
• Confusing filters with UI removal

2. Which of the following is the correct way to represent a filter for price less than $100 in a query parameter?

easy

A. price>100

B. price<100

C. price=100

D. price!=100

Solution

1. Step 1: Understand comparison operators in queries

   The symbol '<' means less than, so 'price<100' filters prices below 100.

2. Step 2: Eliminate incorrect operators

   '>' means greater than, '=' means equal, '!=' means not equal, so they don't match 'less than 100'.

3. Final Answer:

   price<100 -> Option B

4. Quick Check:

   Less than operator = A [OK]

Hint: Use '<' for less than in filters [OK]

Common Mistakes:

• Using '>' instead of '<' for less than
• Confusing '=' with less than
• Using '!=' which means not equal

3. Consider a search system that indexes products by category and price. If a user searches with filters category='books' and price < 20, which data structure best supports fast filtering?

medium

A. A hash map keyed by category with sorted price lists

B. A single unsorted list of all products

C. A queue of products ordered by insertion time

D. A stack of products sorted by price

Solution

1. Step 1: Analyze filtering needs

   Filtering by category and price requires quick lookup by category and efficient price range queries.

2. Step 2: Choose data structure supporting these queries

   A hash map keyed by category allows fast category lookup; sorted price lists enable quick filtering by price.

3. Final Answer:

   A hash map keyed by category with sorted price lists -> Option A

4. Quick Check:

   Hash map + sorted list = B [OK]

Hint: Use hash map for categories and sorted lists for range filters [OK]

Common Mistakes:

• Using unsorted lists causing slow searches
• Using queue or stack which don't support efficient filtering
• Ignoring the need for sorting by price

4. A search filter system is returning incorrect results when filtering by date range. Which of the following is the most likely cause?

medium

A. Date values are stored as strings and compared lexicographically

B. The filter uses numeric comparison on date objects

C. The database index is on the wrong column

D. The search query is missing a filter parameter

Solution

1. Step 1: Understand date comparison issues

   Comparing dates stored as strings can cause wrong order because string comparison is lexicographic.

2. Step 2: Identify why this causes incorrect results

   Dates like '12/01/2023' and '02/12/2023' compared as strings may not sort correctly, causing wrong filter results.

3. Final Answer:

   Date values are stored as strings and compared lexicographically -> Option A

4. Quick Check:

   String date comparison causes errors = A [OK]

Hint: Store dates as date objects, not strings [OK]

Common Mistakes:

• Assuming numeric comparison works on strings
• Ignoring index relevance
• Thinking missing filter param causes wrong filtered results

5. You are designing a scalable search and filter system for an e-commerce site with millions of products. Which approach best balances fast search and flexible filtering?

hard

A. Load all products into memory and filter using loops

B. Store all products in a single SQL table and scan it for every search

C. Use a simple key-value store without indexes

D. Use a distributed search engine with inverted indexes and faceted filters

Solution

1. Step 1: Consider scalability and performance needs

   Millions of products require fast, scalable search with flexible filters.

2. Step 2: Evaluate options for search and filtering

   Distributed search engines with inverted indexes enable fast text search; faceted filters allow flexible attribute filtering efficiently.

3. Step 3: Eliminate inefficient approaches

   Scanning large SQL tables or in-memory filtering is slow and not scalable; key-value stores lack complex search capabilities.

4. Final Answer:

   Use a distributed search engine with inverted indexes and faceted filters -> Option D

5. Quick Check:

   Distributed search + faceted filters = D [OK]

Hint: Use distributed search with faceted filters for scale [OK]

Common Mistakes:

• Relying on full table scans for large data
• Ignoring indexing for search speed
• Using memory-heavy filtering for millions of items