Bird
Raised Fist0

Design an AI Code Assistant - RAG Architecture, Code Indexing, Latency Budget

HLDhardInterview freq: medium
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/6
API Gateway / Proxy

Receive user code query

Accepts WebSocket request with code query payload

Protocol:WebSocket
BeforeUser has typed a code query in client UI
AfterQuery received and forwarded to Query Router
Query Router / Load Balancer

Route query to Retrieval Service

Selects Retrieval Service instance based on load

Protocol:Internal RPC
BeforeQuery Router has incoming query from API Gateway
AfterQuery dispatched to Retrieval Service
Retrieval Service

Lookup code snippets in Code Index Database

Checks Redis cache; on miss queries Code Index DB

Protocol:Redis GET, then DB query
BeforeRetrieval Service has query but no cached results
AfterRelevant code snippets retrieved from Code Index DB
Generation Service

Generate AI response using retrieved snippets

Combines query and snippets to produce answer within latency budget

Protocol:Internal RPC
BeforeGeneration Service has query and code snippets
AfterAI-generated response ready for delivery
API Gateway / Proxy

Send AI response back to user

Pushes response over WebSocket to client UI

Protocol:WebSocket
BeforeResponse ready at API Gateway
AfterUser receives AI code assistant answer
Indexing Worker

Process new code changes from Kafka

Consumes code update events and updates Code Index DB

Protocol:Kafka
BeforeNew code changes published to Kafka topic
AfterCode Index Database updated with latest code
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────────┐
│ API Gateway / Proxy   │
└─────────────┬─────────┘
              │ WebSocket
              ▼
┌───────────────────────────┐
│ Query Router / LB         │
└─────────────┬─────────────┘
              │ Internal RPC
              ▼
┌───────────────────────┐
│ Retrieval Service     │
└───────┬───────────────┘
        │ Redis GET (cache miss)
        ▼
┌───────────────────────────┐
│ Code Index DB             │
└─────────────┬─────────────┘
              │ Kafka (index updates)
              ▼
┌───────────────────────┐
│ Indexing Worker       │
└─────────────┬─────────┘
              │ Kafka (code changes)
              ▼
┌───────────────────────────┐
│ Generation Service         │
└─────────────┬─────────────┘
              │ Internal RPC
              ▼
┌───────────────────────┐
│ API Gateway / Proxy   │
└───────────────────────┘
Design Rationale

This AI Code Assistant architecture uses a Retrieval-Augmented Generation (RAG) pattern to combine indexed code snippets with AI generation. The API Gateway accepts user queries and routes them through a Query Router to the Retrieval Service, which fetches relevant code snippets from the Code Index Database. The Generation Service then produces an AI response using these snippets, respecting latency budgets for responsiveness. Code indexing is asynchronously updated by the Indexing Worker consuming Kafka events from code changes, ensuring fresh data without blocking query paths. Kafka acts as the event backbone for indexing updates, decoupling ingestion from query serving. This vertical flow ensures low latency and scalability by separating retrieval and generation concerns.

Primary Focus

✓

Practice

(1/5)
1. In the request flow, at Step 4 the Cache Node receives a key lookup request. What happens if the key is not found in the local cache at this step?
easy
A. The Cache Node immediately returns a cache miss to the client
B. The Cache Node forwards the request to the next node in the consistent hash ring
C. The Cache Node broadcasts the request to all cache nodes to find the key
D. The Cache Node fetches the value from the backing data store and updates its cache

Solution

  1. Step 1: Trace cache miss handling at Step 4

    On a miss, the node does not return immediately; it attempts to find the key elsewhere.

  2. Step 2: Broadcast vs forwarding

    Forwarding to next node breaks consistent hashing guarantees; fetching from backing store happens after confirming absence in cache cluster.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Cache node broadcasts to find key before fallback [check]
Hint: Cache miss triggers broadcast to find key in cluster [OK]
Common Mistakes:
2. In the newsfeed system, at Step 4 the Delivery Worker reads a fan-out event from Kafka and attempts to deliver it to followers. What happens if a follower's device is offline at this step?
easy
A. Delivery Worker marks the message as sent but not delivered, queues for retry later
B. Delivery Worker drops the message to avoid blocking other deliveries
C. Delivery Worker retries immediately until the follower is online
D. Delivery Worker pushes the message to the follower's device cache directly

Solution

  1. Step 1: Trace Delivery Worker behavior at Step 4

    The Delivery Worker reads fan-out events from Kafka and attempts delivery. If the follower is offline, immediate retry (A) would block and reduce throughput. Dropping messages (C) causes data loss. Pushing directly to device cache (D) is impossible if offline.

  2. Step 2: Understand retry mechanism

    The system marks the message as sent but not delivered, queues it for retry later when the follower reconnects, ensuring eventual delivery without blocking.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Offline followers cause queued retries, not immediate blocking [check]
Hint: Offline followers cause queued retries, not immediate blocking [check]
Common Mistakes:
3. A collaborative document editing system must ensure that concurrent edits from multiple users are merged in real-time without conflicts or data loss. Which component is primarily responsible for resolving these concurrent edits to maintain consistency?
easy
A. A relational database with row-level locking to serialize edits
B. An Operational Transformation (OT) or CRDT engine that transforms or merges concurrent operations
C. A distributed cache like Redis to store the latest document state
D. A message queue that orders all edit events strictly by timestamp

Solution

  1. Step 1: Identify the role of conflict resolution in collaborative editing

    Concurrent edits from multiple users require a mechanism to merge changes without overwriting or losing data. Simple serialization or ordering is insufficient due to network delays and offline edits.

  2. Step 2: Understand why OT or CRDT is needed

    Operational Transformation (OT) and Conflict-free Replicated Data Types (CRDTs) are designed to transform or merge concurrent operations in real-time, ensuring eventual consistency without central locking.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    OT/CRDT engines merge concurrent edits without locking or strict ordering [check]
Hint: OT/CRDT merge concurrent edits without locking or strict order [OK]
Common Mistakes:
4. Without the message broker's offset commit mechanism, what exactly breaks in the delivery flow?
medium
A. Producers block indefinitely waiting for acknowledgments
B. Messages are lost silently because offsets are not persisted
C. Messages are duplicated indefinitely because consumers cannot track progress
D. Consumers receive messages out of order due to missing broker coordination

Solution

  1. Step 1: Role of offset commit in consumer progress tracking

    Offset commits mark messages as processed; without them, consumers reprocess the same messages repeatedly.

  2. Step 2: Consequence of missing offset commits

    Consumers cannot advance their read position, causing duplicate deliveries; messages are not lost or reordered inherently.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    No offset commit -> infinite duplicates, no progress [check]
Hint: No offset commit -> consumer reprocesses same messages repeatedly [OK]
Common Mistakes:
5. At 10x traffic scale, which component in a load balanced web service is most likely to fail first, and why?
hard
A. The DNS server, due to increased domain resolution requests
B. The load balancer, because it maintains stateful TCP connections and cannot shard easily
C. The backend servers, because they cannot horizontally scale beyond a fixed number
D. The database, due to increased query volume and connection limits

Solution

  1. Step 1: Analyze component scaling characteristics

    Backend servers (The backend servers, because they cannot horizontally scale beyond a fixed number) typically scale horizontally. DNS servers (The DNS server, due to increased domain resolution requests) are highly distributed and cacheable. Load balancers (The load balancer, because it maintains stateful TCP connections and cannot shard easily) can be scaled with multiple instances and sharding.

  2. Step 2: Identify database bottleneck

    Databases (The database, due to increased query volume and connection limits) often face connection limits and query volume bottlenecks first at large scale, requiring sharding or caching to scale further.

  3. Step 3: Confirm failure mechanism

    Without proper scaling, the database becomes the first component to fail under 10x load due to resource exhaustion.

  4. Final Answer:

    Option D -> Option D

  5. Quick Check:

    Database connection limits cause first failure at 10x scale [check]
Hint: Database connection limits break first at scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard