Bird
Raised Fist0

Full-Text Search Architecture - Inverted Index Design from Scratch

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/5
Kafka Cluster

Receives new document update event

Document content is published as a Kafka message

Protocol:Kafka Publish
BeforeNew document exists only on client side
AfterDocument update event durably stored in Kafka topic
Indexing Service

Consumes document update from Kafka

Parses document and extracts terms for inverted index

Protocol:Kafka Consume
BeforeKafka topic contains unprocessed document update
AfterInverted index entries generated for document terms
Storage Layer (Cassandra)

Stores inverted index entries

Writes term-to-document mappings into Cassandra tables

Protocol:CQL Write
BeforeInverted index entries exist only in memory
AfterInverted index data durably stored in Cassandra
Query Router

Receives user search query

Parses query and routes to relevant search shards

Protocol:HTTP/REST
BeforeUser query received at front-end
AfterQuery forwarded to appropriate Query Service shards
Query Service / Search Shards

Executes search on inverted index

Fetches matching documents from Cassandra and ranks results

Protocol:RPC / CQL Read
BeforeQuery received from Query Router
AfterSearch results generated and returned to user
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────────┐
│    Kafka Cluster      │
│  (Message Queue for   │
│   document updates)   │
└──────────┬────────────┘
           │ Kafka Publish
           ▼
┌───────────────────────┐
│   Indexing Service    │
│ (Consumes Kafka msgs, │
│  builds inverted idx) │
└──────────┬────────────┘
           │ Write inverted index
           ▼
┌───────────────────────────────┐
│ Storage Layer (Cassandra)     │
│ (Stores inverted index data)  │
└──────────────┬────────────────┘
               │ Query request
               ▼
┌───────────────────────┐
│     Query Router      │
│ (Routes queries to    │
│  appropriate shards)  │
└──────────┬────────────┘
           │ RPC Query
           ▼
┌───────────────────────────────┐
│ Query Service / Search Shards  │
│ (Executes search on index data)│
└───────────────────────────────┘
Design Rationale

This architecture uses Kafka as a durable event queue to capture document updates, ensuring reliable ingestion. The Indexing Service consumes these events to build an inverted index, which is stored in Apache Cassandra for scalable, distributed storage. When a user issues a search query, the Query Router directs the request to the appropriate Query Service shards that hold partitions of the index. These shards execute the search by querying Cassandra and return ranked results. This design decouples ingestion from querying, supports horizontal scaling, and ensures low-latency search by partitioning the index across shards.

Primary Focus

✓

Practice

(1/5)
1. A high-traffic e-commerce platform expects a sudden surge in user requests immediately after launch. Which component is best suited to pre-populate the cache with frequently accessed product data to avoid cold-start latency?
easy
A. Implement a cache warming service that preloads hot keys into the in-memory cache before launch
B. Use a CDN to cache static assets only, ignoring dynamic product data
C. Rely on lazy loading where cache entries are populated only on first user request
D. Use a write-through cache that updates cache only after database writes occur

Solution

  1. Step 1: Identify cache warming purpose

    Cache warming service proactively loads hot keys to avoid cold cache latency at launch.

  2. Step 2: Eliminate lazy and write-through options

    Lazy loading causes initial latency; write-through updates cache reactively, not proactively.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Preloading hot keys before launch avoids cold cache misses [check]
Hint: Preload hot keys before launch to avoid cold cache misses [OK]
Common Mistakes:
2. In the request flow, at step 5 the Intent Classifier component receives a user query. What happens if the confidence score is below the threshold at this step?
easy
A. The query is logged and a fallback notification is triggered asynchronously
B. The query is immediately sent to the human agent without further processing
C. The query is discarded and the user is asked to rephrase
D. The query is forwarded to the Fallback Handler component for alternative resolution

Solution

  1. Step 1: Trace Intent Classifier decision at Step 5

    When confidence is low, the system logs the query and triggers fallback notification asynchronously to avoid blocking.

  2. Step 2: Confirm fallback handling behavior

    Immediate human routing (A) blocks scalability. Forwarding directly (B) is plausible but synchronous. Discarding (C) harms UX. Logging and async notification (D) balances reliability and responsiveness.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Low confidence -> async fallback notification and logging [check]
Hint: Low confidence triggers async fallback notification [check]
Common Mistakes:
3. In a REST API request flow, at Step 4 the API Gateway validates the HTTP method and resource path. What happens if the client sends a DELETE request to a resource that does not exist?
easy
A. The API Gateway immediately returns HTTP 404 Not Found without backend call
B. The API Gateway returns HTTP 200 OK assuming idempotent delete
C. The API Gateway returns HTTP 405 Method Not Allowed because DELETE is unsupported
D. The API Gateway forwards the request to the backend service without validation

Solution

  1. Step 1: Understand API Gateway role at Step 4

    It validates method and resource existence before forwarding.

  2. Step 2: Recognize that DELETE on non-existent resource returns 404

    REST semantics specify 404 if resource is missing, preventing unnecessary backend calls.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    API Gateway returns 404 for DELETE on missing resource [check]
Hint: DELETE missing resource -> 404 Not Found [check]
Common Mistakes:
4. If the hash functions used in the Count-Min Sketch become non-independent and correlated, what exact failure occurs in frequency estimation?
medium
A. The sketch fails to increment counters, causing zero counts
B. The sketch counters become negative due to hash collisions
C. Frequency estimates become biased and error bounds no longer hold
D. The system crashes due to infinite loops in hashing

Solution

  1. Step 1: Understand hash function independence role

    Independent hash functions ensure error bounds and unbiased frequency estimates.

  2. Step 2: Effect of correlated hash functions

    Correlation causes counters to overlap more than expected, biasing estimates and invalidating error guarantees.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Correlated hashes bias frequency estimates, breaking error guarantees [check]
Hint: Independent hashes ensure unbiased estimates; correlation breaks guarantees [check]
Common Mistakes:
5. If the rollback mechanism is missing in the deployment system, what exact failure occurs when a faulty version is released during Canary deployment?
medium
A. Users experience degraded service indefinitely as faulty version remains live without fallback
B. The previous stable version is restored manually without affecting users
C. The deployment system retries indefinitely, causing deployment pipeline blockage
D. The system automatically disables all new features, causing partial outages

Solution

  1. Step 1: Understand rollback absence impact

    Without rollback, faulty Canary version cannot be reverted automatically.

  2. Step 2: Trace failure effect

    Faulty version remains live, causing degraded user experience indefinitely.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    No rollback -> faulty version stays live -> degraded service persists [check]
Hint: No rollback means faulty version stays live, degrading users indefinitely [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard