Search and metadata in HLD - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Architecture

intermediate

Designing a scalable search system for metadata-rich documents

You need to design a search system that indexes documents with rich metadata (tags, authors, dates). Which architecture best supports fast, scalable search with frequent metadata updates?

A Use a distributed search engine like Elasticsearch with separate indices for metadata and content.

B Use a relational database with full-text search and update metadata in place.

C Store documents in a NoSQL key-value store and perform search by scanning all keys.

D Use a file system with metadata stored in separate JSON files and search by reading files sequentially.

💡 Hint

Think about systems optimized for search and handling frequent updates efficiently.

Distributed search engines like Elasticsearch are designed for fast, scalable search and can handle frequent metadata updates by indexing separately. Relational databases are less efficient for large-scale full-text search. Scanning all keys or files sequentially is slow and not scalable.

❓ scaling

intermediate

Estimating capacity for a metadata search service

Your metadata search service expects 10 million documents with an average of 20 metadata fields each. You expect 1000 queries per second. What is the best way to estimate the required hardware capacity?

A Estimate index size for metadata, average query complexity, and use benchmarks of similar systems to size CPU and memory.

B Calculate storage size for all metadata and content, then multiply by query rate to get CPU needs.

C Count number of documents only and assign one CPU core per 1000 documents.

D Ignore metadata size and focus only on network bandwidth for query traffic.

💡 Hint

Consider both data size and query complexity, and use real-world benchmarks.

Estimating capacity requires understanding index size, query complexity, and benchmarking similar systems. Simple document counts or ignoring metadata size leads to inaccurate estimates.

❓ tradeoff

advanced

Choosing between inverted index and graph database for metadata search

You must choose a data structure for searching documents by metadata. Which option best fits a use case with complex relationships between metadata (e.g., authors collaborating, hierarchical tags)?

A Use a relational database with multiple join tables for metadata relationships.

B Use an inverted index optimized for keyword search across metadata fields.

C Use a simple key-value store with metadata serialized as JSON strings.

D Use a graph database to model and query complex relationships between metadata entities.

💡 Hint

Consider which data structure naturally represents relationships and supports complex queries.

Graph databases are designed to model and query complex relationships efficiently. Inverted indexes excel at keyword search but not relationship queries. Relational joins can be expensive at scale. Key-value stores lack query flexibility.

🧠 Conceptual

advanced

Understanding metadata freshness impact on search results

In a search system with frequent metadata updates, what is the main tradeoff when choosing between real-time indexing and batch indexing?

A Real-time indexing reduces query latency but increases network bandwidth; batch indexing reduces bandwidth usage.

B Batch indexing improves freshness but requires more storage; real-time indexing reduces storage needs.

C Real-time indexing improves freshness but increases system load; batch indexing reduces load but delays updates.

D Batch indexing allows partial updates; real-time indexing requires full reindexing each time.

💡 Hint

Think about how update frequency affects system performance and data freshness.

Real-time indexing keeps data fresh but adds load due to constant updates. Batch indexing reduces load by updating less often but causes delays in reflecting changes.

❓ component

expert

Request flow in a distributed metadata search system

Trace the request flow when a user searches for documents by metadata in a distributed search system with multiple shards and a metadata cache layer. Which sequence correctly describes the flow?

A 1, 3, 4, 2, 5, 6

B 1, 2, 3, 4, 5, 6

C 1, 2, 4, 3, 5, 6

D 1, 3, 2, 4, 5, 6

💡 Hint

Consider cache lookup before querying shards and updating cache after aggregation.

The correct flow starts with user query to API gateway, cache check, forwarding to shards on miss, shards returning partial results, aggregation, returning results, and cache update.

Practice

1. What is the primary purpose of metadata in a search system?

easy

A. To display images on the website

B. To store user passwords securely

C. To describe data and make search faster

D. To manage network connections

Solution

1. Step 1: Understand metadata role

   Metadata provides information about data, like tags or descriptions.

2. Step 2: Connect metadata to search

   Search engines use metadata to quickly find relevant data without scanning everything.

3. Final Answer:

   To describe data and make search faster -> Option C

4. Quick Check:

   Metadata = Data description for search [OK]

Hint: Metadata helps find data faster by describing it [OK]

Common Mistakes:

• Confusing metadata with user data
• Thinking metadata stores passwords
• Assuming metadata manages network

2. Which of the following is a correct example of metadata used in search?

easy

A. title = 'Introduction to Cats'

B. file_size = 2048

C. user_password = '1234'

D. connection_timeout = 30

Solution

1. Step 1: Identify metadata examples

   Metadata describes content, like titles, tags, or dates.

2. Step 2: Check each option

   title = 'Introduction to Cats' shows a title, which is metadata describing content. Others are config or sensitive data.

3. Final Answer:

   title = 'Introduction to Cats' -> Option A

4. Quick Check:

   Title is metadata for search [OK]

Hint: Metadata describes content, not configs or passwords [OK]

Common Mistakes:

• Choosing config values as metadata
• Confusing sensitive data with metadata
• Ignoring descriptive fields

3. Given a search system with metadata index, what is the expected output when searching for "apple" if metadata contains {"title": "apple pie", "tags": ["fruit", "dessert"]}?

medium

A. No results found

B. Returns item with title "apple pie"

C. Returns all items with tag "fruit" only

D. Returns items with tag "dessert" only

Solution

1. Step 1: Understand search with metadata

   Search looks for matches in metadata fields like title and tags.

2. Step 2: Check if "apple" matches metadata

   "apple" matches the title "apple pie", so the item is returned.

3. Final Answer:

   Returns item with title "apple pie" -> Option B

4. Quick Check:

   Search matches title containing "apple" [OK]

Hint: Search matches metadata fields containing query word [OK]

Common Mistakes:

• Ignoring title field in search
• Returning unrelated tags only
• Assuming no results if exact match missing

4. A search system's metadata index is not returning expected results. Which issue below is most likely the cause?

medium

A. Database password is incorrect

B. User interface colors are dull

C. Network cables are unplugged

D. Metadata is not updated after data changes

Solution

1. Step 1: Identify cause of search failure

   If metadata is stale, search index won't reflect latest data.

2. Step 2: Evaluate options

   Only Metadata is not updated after data changes relates to metadata and search correctness; others are unrelated.

3. Final Answer:

   Metadata is not updated after data changes -> Option D

4. Quick Check:

   Stale metadata breaks search results [OK]

Hint: Keep metadata updated to ensure correct search [OK]

Common Mistakes:

• Blaming UI or network for search logic errors
• Ignoring metadata update process
• Confusing unrelated system issues

5. You are designing a scalable search system for millions of users. Which approach best ensures fast search using metadata?

hard

A. Use distributed indexing with metadata shards and update indexes asynchronously

B. Store metadata in a centralized database and scan all records on each search

C. Keep metadata only on user devices and search locally

D. Disable metadata to reduce storage and search raw data only

Solution

1. Step 1: Understand scalability needs

   Millions of users require fast, distributed search to avoid bottlenecks.

2. Step 2: Evaluate options for scalability

   Use distributed indexing with metadata shards and update indexes asynchronously uses distributed indexing and async updates, which scales well and keeps search fast.

3. Final Answer:

   Use distributed indexing with metadata shards and update indexes asynchronously -> Option A

4. Quick Check:

   Distributed indexing + async updates = scalable search [OK]

Hint: Distribute metadata index and update asynchronously for scale [OK]

Common Mistakes:

• Scanning all data centrally causes slow search
• Relying on local device metadata limits scale
• Disabling metadata removes search efficiency