Bird
Raised Fist0

Why Async Messaging and What Synchronous Calls Cannot Handle

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
▶
Request Trace
Step 1/6
Producers

Publish event

Producer sends event to Kafka asynchronously

Protocol:Kafka Producer API
BeforeEvent exists only in Producer's memory
AfterEvent is durably stored in Kafka partition
Kafka Cluster

Store event

Kafka appends event to partition log and replicates

Protocol:Internal Kafka replication
BeforePartition log ends at offset N
AfterPartition log ends at offset N+1 with new event
Consumer Service

Poll and consume event

Consumer reads event from Kafka asynchronously

Protocol:Kafka Consumer API
BeforeEvent is unread in Kafka partition
AfterEvent marked as consumed; Consumer processes event
Consumer Service

Invoke downstream

Consumer calls Downstream Microservices asynchronously

Protocol:REST/gRPC
BeforeConsumer has event data ready to process
AfterDownstream Microservices receive request for processing
Downstream Microservices

Process request

Downstream services execute business logic and update state

Protocol:Internal service logic
BeforeRequest received but not processed
AfterRequest processed; state updated accordingly
ZooKeeper Ensemble

Coordinate Kafka cluster

ZooKeeper manages Kafka leader election and metadata

Protocol:ZooKeeper protocol
BeforeKafka cluster metadata may be stale
AfterKafka cluster metadata is consistent and up-to-date
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────┐
│   Producers   │
└───────┬───────┘
        │ Kafka Producer API
        ▼
┌───────────────┐
│ Kafka Cluster │
│  (Partitions) │
└───────┬───────┘
        │ Kafka Consumer API
        ▼
┌───────────────────┐
│ Consumer Service  │
│ (Async Processing)│
└───────┬───────────┘
        │ REST/gRPC
        ▼
┌───────────────────────┐
│ Downstream Microservs │
│ (Business Logic & DB) │
└───────────────────────┘
        ▲
        │ ZooKeeper Coord
┌───────────────┐
│ZooKeeper Ensmb│
└───────────────┘

This architecture demonstrates why asynchronous messaging via Kafka is essential when synchronous calls cannot handle high throughput or decoupling. Producers publish events asynchronously to Kafka, which durably stores them in partitions managed by ZooKeeper. Consumer Services poll Kafka independently, processing events at their own pace and invoking downstream microservices asynchronously. This decoupling allows the system to handle spikes and failures gracefully, unlike synchronous calls that block and cascade failures. ZooKeeper ensures Kafka cluster coordination and metadata consistency, enabling reliable event streaming. Overall, asynchronous messaging improves scalability, fault tolerance, and responsiveness in distributed systems.

Primary Focus

✓

Practice

(1/5)
1. A client application needs to fetch deeply nested related data in a single request without multiple round-trips. Which API design approach best supports this requirement?
easy
A. Use GraphQL to request exactly the nested data needed in one query
B. Use WebSockets to stream nested data continuously
C. Use REST with query parameters to fetch all nested data in one endpoint
D. Use REST endpoints with multiple sequential calls to fetch nested resources

Solution

  1. Step 1: Identify nested data fetching needs

    REST typically requires multiple calls or over-fetching to get nested data, causing inefficiency.
  2. Step 2: Recognize GraphQL's single query nested fetch

    GraphQL allows clients to specify nested fields in one request, reducing round-trips and over-fetching.
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    GraphQL single query fetches nested data efficiently [check]
Hint: GraphQL fetches nested data in one query [check]
Common Mistakes:
2. If the Gossip Protocol component fails to propagate membership updates, what is the most immediate failure observed in the key-value store?
medium
A. Consistent hashing ring fails to rebalance keys on node failure
B. Nodes continue to send requests to failed nodes causing timeouts
C. Clients receive stale data due to vector clock version conflicts
D. Write operations block indefinitely waiting for quorum acknowledgments

Solution

  1. Step 1: Understand gossip failure impact

    Without membership updates, nodes are unaware of failed nodes and continue routing requests to them.
  2. Step 2: Eliminate other failure modes

    Vector clocks handle versioning, unrelated to gossip failure. Consistent hashing ring updates rely on membership info but do not fail immediately. Write blocking on quorum is unrelated to gossip propagation.
  3. Final Answer:

    Option B -> Option B
  4. Quick Check:

    Missing membership updates cause requests to failed nodes causing timeouts [check]
Hint: Gossip failure -> nodes unaware of failures, keep sending requests [OK]
Common Mistakes:
3. For handling concurrent seat bookings, which architecture better suits a high-traffic ticketing system requiring low latency and strong consistency?
medium
A. Use a centralized relational database with serializable transactions to lock seats
B. Use client-side seat locking with server-side eventual consistency checks
C. Use a message queue to serialize all booking requests for each seat
D. Use a distributed cache with optimistic locking and retry logic for seat reservation

Solution

  1. Step 1: Analyze concurrency and latency requirements

    High traffic and low latency require minimizing blocking and centralized bottlenecks.
  2. Step 2: Evaluate architectures

    Centralized DB serializable transactions cause contention and latency; message queue serialization adds delay; client-side locking is unreliable. Distributed cache with optimistic locking allows concurrent attempts with retries, balancing consistency and performance.
  3. Final Answer:

    Option D -> Option D
  4. Quick Check:

    Optimistic locking in distributed cache balances concurrency and consistency [check]
Hint: Optimistic locking + retry beats centralized locks under load [OK]
Common Mistakes:
4. For ranking answers in the feed, which approach is better given the need for real-time updates and complex ranking signals including user behavior and spam scores?
medium
A. Use batch processing with daily recomputation of rankings in a data warehouse
B. Use client-side ranking to reduce server load and latency
C. Use a simple heuristic ranking computed at write time with no model updates
D. Use an online feature store with real-time scoring and incremental model updates

Solution

  1. Step 1: Evaluate batch vs real-time ranking

    Batch (A) is too stale for real-time user behavior; heuristic (C) lacks adaptability; client-side (D) cannot access server-side signals securely.
  2. Step 2: Online feature store benefits

    Online feature stores enable real-time scoring with fresh data and incremental model updates, supporting complex signals and spam detection.
  3. Final Answer:

    Option D -> Option D
  4. Quick Check:

    Online feature store supports real-time, complex ranking [check]
Hint: Real-time ranking needs online features, not stale batch [check]
Common Mistakes:
5. At 10x traffic scale, which API Gateway component is most likely to become the bottleneck first, and why?
hard
A. Rate limiting store, because centralized counters cause contention and latency spikes
B. Authentication service, because token validation is CPU-intensive and stateful
C. API Gateway routing logic, because routing tables grow exponentially with services
D. Backend microservices, because they cannot horizontally scale as easily as the gateway

Solution

  1. Step 1: Identify scaling challenges

    Rate limiting often relies on centralized counters or stores (e.g., Redis) that become contention points under high request rates.
  2. Step 2: Eliminate other options

    Backend microservices, because they cannot horizontally scale as easily as the gateway is incorrect as backend services typically scale horizontally. API Gateway routing logic, because routing tables grow exponentially with services is unlikely as routing tables are cached and optimized. Authentication service, because token validation is CPU-intensive and stateful is less likely as token validation is stateless and can be cached.
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    Centralized rate limiting store bottlenecks first at scale [check]
Hint: Centralized counters bottleneck rate limiting at scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard