Bird
Raised Fist0

Faceted Search and Filtering at Scale

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/8
API Gateway

Receives faceted search query from user

User submits HTTP REST request with filters and facets

Protocol:HTTP/REST
BeforeUser has composed search query on client device
AfterAPI Gateway has received and parsed the search query
Query Router

Routes query to Facet Config Service and Elasticsearch

Sends gRPC requests to fetch facet metadata and execute search

Protocol:gRPC
BeforeQuery Router has raw search query from API Gateway
AfterQuery Router has initiated facet metadata fetch and search execution
Facet Config Service

Fetches facet metadata from Metadata Store (Cassandra)

Queries Cassandra for current facet configurations

Protocol:Cassandra CQL
BeforeFacet Config Service has received request for facet metadata
AfterFacet Config Service has retrieved up-to-date facet metadata
Metadata Store (Cassandra)

Returns facet metadata to Facet Config Service

Serves latest facet config data for query enrichment

Protocol:Cassandra CQL
BeforeMetadata Store (Cassandra) has stored latest facet configs
AfterMetadata Store (Cassandra) has responded with facet metadata
Indexing Pipeline (Kafka + Log Processor)

Consumes Kafka logs and updates Elasticsearch index

Processes data changes and pushes bulk updates to Elasticsearch

Protocol:Kafka Consumer + Elasticsearch Bulk API
BeforeKafka queue contains new data change logs
AfterElasticsearch index is updated with latest data for search
Elasticsearch Cluster

Executes faceted search query with metadata

Combines user filters and facet metadata to return results

Protocol:Elasticsearch Query API
BeforeElasticsearch has latest indexed data and facet metadata
AfterElasticsearch returns search results with facet counts
Query Router

Aggregates results and facet metadata, sends response

Combines Elasticsearch results and facet info for API Gateway

Protocol:gRPC
BeforeQuery Router has received search results and facet metadata
AfterQuery Router has sent combined response to API Gateway
API Gateway

Returns faceted search results to user

Sends HTTP REST response with search hits and facet counts

Protocol:HTTP/REST
BeforeAPI Gateway has combined search and facet data
AfterUser receives search results with facets on client device
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────┐
│  API Gateway  │
└───────┬───────┘
        │ HTTP/REST
        ▼
┌───────────────┐
│ Query Router  │
└───────┬───────┘
        │ gRPC
        ▼
┌───────────────────────────────┐
│ Facet Config Service           │
│ (fetches facet metadata)       │
└───────────────┬───────────────┘
                │ Cassandra CQL
                ▼
      ┌─────────────────────┐
      │ Metadata Store      │
      │ (Cassandra)         │
      └─────────┬───────────┘
                │ Kafka Consumer
                ▼
┌───────────────────────────────┐
│ Indexing Pipeline             │
│ (Kafka + Log Processor)       │
└───────────────┬───────────────┘
                │ Elasticsearch Bulk API
                ▼
      ┌─────────────────────┐
      │ Elasticsearch       │
      │ Cluster             │
      └─────────────────────┘
Design Rationale

This architecture supports faceted search at scale by separating query routing, facet metadata retrieval, and search execution. The API Gateway accepts user queries and forwards them to the Query Router, which concurrently fetches facet configurations from the Facet Config Service backed by Cassandra and executes the search on Elasticsearch. The Indexing Pipeline consumes Kafka logs to keep Elasticsearch indexes fresh. This design ensures up-to-date facet metadata and search results with low latency. The vertical flow emphasizes clear separation of concerns and asynchronous indexing, enabling scalable, real-time faceted search.

Primary Focus

✓

Practice

(1/5)
1. In the live streaming pipeline, at step 4 the Transcoder Worker reads packets from Kafka and generates HLS segments. What happens if the Transcoder Worker encounters a delayed packet for a segment already finalized?
easy
A. The Transcoder discards the packet and continues streaming
B. The Transcoder reopens the finalized segment and appends the packet
C. The Transcoder buffers the packet until the next segment window
D. The Transcoder triggers a full stream restart to maintain consistency

Solution

  1. Step 1: Understand segment finalization

    Once an HLS segment is finalized and published, it is immutable to ensure consistent playback for viewers.

  2. Step 2: Handling delayed packets

    Reopening finalized segments (B) breaks immutability. Buffering for next segment (C) misplaces packets out of order. Restarting stream (D) is costly and unnecessary. Discarding late packets (A) maintains low latency and stream integrity.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Discarding late packets preserves segment immutability and low latency [check]
Hint: Finalized HLS segments are immutable; discard late packets [check]
Common Mistakes:
2. If the Kafka broker cluster loses ZooKeeper coordination, what exactly breaks in the message delivery process?
medium
A. Messages are lost immediately as brokers cannot write to logs
B. Producers can still write, but partition leadership cannot be elected, causing delivery stalls
C. Consumers receive duplicate messages due to offset corruption
D. The entire cluster shuts down automatically to prevent data loss

Solution

  1. Step 1: Role of ZooKeeper in Kafka

    ZooKeeper manages partition leadership and consumer offsets; losing it affects coordination but not immediate message loss.

  2. Step 2: Failure impact on delivery

    Without ZooKeeper, offset metadata can become inconsistent, causing consumers to re-read messages and receive duplicates (C); producers may still write (B is partially true but leadership election stalls eventually); cluster does not shut down automatically (D).

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    ZooKeeper loss -> offset inconsistency -> duplicate consumer messages [check]
Hint: ZooKeeper loss corrupts offsets causing duplicates, not immediate loss [check]
Common Mistakes:
3. For the For-You feed algorithm, which architecture best balances freshness of recommendations with scalability when serving millions of users?
medium
A. Use a monolithic service that recomputes recommendations synchronously on every user request
B. Use a client-side recommendation engine to reduce server load
C. Use an offline batch job to precompute recommendations periodically and serve from cache
D. Use a real-time streaming pipeline to update recommendations instantly per user action

Solution

  1. Step 1: Evaluate synchronous recomputation

    Monolithic synchronous (A) causes high latency and poor scalability.

  2. Step 2: Compare batch vs streaming

    Streaming (C) offers freshness but is complex and costly at scale; client-side (D) lacks data for quality recommendations.

  3. Step 3: Select batch precompute

    Batch jobs (B) balance freshness and scalability by precomputing and caching recommendations.

  4. Final Answer:

    Option C -> Option C

  5. Quick Check:

    Batch precompute caches recommendations for scalable freshness [check]
Hint: Batch precompute balances freshness and scale best [OK]
Common Mistakes:
4. Without the event store (Kafka) in the system, what exactly breaks during command processing?
medium
A. Commands are processed twice causing duplicate events
B. Commands block until query model updates, causing high latency and no durability
C. Commands fail silently without any error notification
D. Commands are queued but never delivered to the read model

Solution

  1. Step 1: Identify event store role in command processing

    The event store (Kafka) durably persists events and decouples command processing from read model updates.

  2. Step 2: Trace failure without event store

    Without Kafka, commands have no durable event persistence, causing silent failures as events are lost without notification.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    No event store -> commands lost silently, no durability [check]
Hint: No event store -> silent command loss, no durability [check]
Common Mistakes:
5. For internal service communication requiring bidirectional streaming with mobile clients that have intermittent connectivity, which approach is best?
medium
A. Use gRPC bidirectional streaming directly for real-time updates
B. Use gRPC unary calls with client-side polling and exponential backoff
C. Use WebSockets with JSON payloads for flexible streaming
D. Use REST long polling to simulate streaming behavior

Solution

  1. Step 1: Analyze client connectivity constraints

    Mobile clients often have intermittent connectivity and poor support for HTTP/2 streaming.

  2. Step 2: Evaluate streaming options

    Direct gRPC streaming (A) is unreliable on mobile. WebSockets (C) add complexity and JSON is less efficient. REST long polling (D) is less efficient and higher latency. Unary calls with polling and backoff (B) handle intermittent connectivity gracefully.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Mobile intermittent -> unary calls + polling with backoff [check]
Hint: Mobile clients -> unary calls + polling, not streaming [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard