Bird
Raised Fist0

Design a Distributed Counter - Sharded Counters, CRDT for Eventual Consistency

HLDhardInterview freq: medium
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/5
API Gateway

Receive increment request

Accept HTTP POST to increment counter with ID

Protocol:HTTP POST
BeforeNo increment request received for this counter
AfterIncrement request validated and ready to route
Write Router

Route increment to shard

Determine shard from counter ID and forward increment

Protocol:Internal RPC
BeforeIncrement request received from API Gateway
AfterIncrement request dispatched to correct shard database
Shard CRDT DBs

Apply CRDT increment

Update local shard count using CRDT merge operation

Protocol:Local DB write
BeforeShard holds previous partial count
AfterShard count updated with new increment, CRDT state merged
Aggregator Service

Aggregate shard counts

Periodically read all shard counts and compute global total

Protocol:Internal RPC
BeforeShard counts updated independently
AfterGlobal count aggregated and ready for caching
Read Cache

Cache aggregated count

Store global count for fast read queries

Protocol:Cache write
BeforeNo cached global count or stale value
AfterRead cache updated with latest aggregated count
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────┐
│  API Gateway  │
└───────┬───────┘
        │ HTTP POST Increment
        ▼
┌───────────────┐
│ Write Router  │
└───────┬───────┘
        │ Route by shard key (counter ID)
        ▼
┌────────────────────┐
│ Shard CRDT DBs     │
│ (multiple shards)  │
└───────┬────────────┘
        │ CRDT merge update
        ▼
┌───────────────────┐
│ Aggregator Service│
└───────┬───────────┘
        │ Aggregate shard counts
        ▼
┌────────────┐
│ Read Cache │
└────────────┘
Design Rationale

This distributed counter system uses sharded CRDT databases to enable high write scalability and eventual consistency. The API Gateway receives increment requests and forwards them to the Write Router, which routes each request to the appropriate shard based on the counter ID. Each shard stores partial counts as CRDTs, allowing concurrent updates without conflicts. The Aggregator Service periodically merges shard counts to compute the global total, which is cached in the Read Cache for fast read access. This architecture balances write scalability with eventual consistency and low-latency reads.

Primary Focus

✓

Practice

(1/5)
1. A video streaming service needs to serve millions of concurrent users with minimal latency and support adaptive bitrate streaming. Which component is primarily responsible for caching and delivering video segments closest to the user?
easy
A. Database that stores user playback history and preferences
B. Origin Server that stores all video content and handles user requests directly
C. Load Balancer that distributes incoming requests evenly across origin servers
D. Edge CDN Node that caches HLS segments and serves them based on user location

Solution

  1. Step 1: Identify the role of Edge CDN Nodes

    Edge CDN nodes cache video segments near users to reduce latency and offload origin servers.

  2. Step 2: Eliminate other options

    Origin servers handle storage but not low-latency delivery; load balancers distribute requests but don't cache content; databases store metadata, not video segments.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Edge CDN caches segments closest to users for low latency [check]
Hint: Edge nodes cache content near users for low latency [check]
Common Mistakes:
2. If the Feature Store's update stream (e.g., Kafka) fails, what exactly breaks in the search ranking system?
medium
A. Model Serving continues scoring with stale features, causing degraded ranking quality
B. Feature Store blocks all read requests until the update stream recovers, causing high latency
C. A/B testing traffic routing fails, causing all users to see the control model only
D. The system loses all feature data permanently, causing model serving to crash

Solution

  1. Step 1: Trace failure impact of update stream loss

    The update stream failure stops feature updates but does not block reads or cause permanent data loss.

  2. Step 2: Identify component relying on update stream

    A/B testing routing depends on fresh feature updates to decide traffic splits; without updates, routing falls back to control model only.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Update stream failure -> A/B routing fallback to control model [check]
Hint: Update stream failure -> A/B routing falls back, not feature reads [check]
Common Mistakes:
3. For Pastebin analytics tracking snippet views in real-time, which architecture best balances scalability and data freshness?
medium
A. Use a streaming platform like Kafka to collect events and process them asynchronously in batches
B. Store all view events directly in a relational database with immediate aggregation queries
C. Log view events to flat files and run daily MapReduce jobs for analytics
D. Send view events directly to a NoSQL store with eventual consistency for real-time queries

Solution

  1. Step 1: Analyze real-time analytics requirements

    High throughput and near real-time aggregation require decoupling ingestion and processing.

  2. Step 2: Evaluate architectures

    Kafka streaming (B) supports scalable event ingestion and asynchronous processing; relational DB (A) can't handle high write load; flat files (C) have high latency; NoSQL eventual consistency (D) may cause stale reads.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka decouples ingestion and processing for scalable real-time analytics [check]
Hint: Stream events asynchronously for scalable real-time analytics [check]
Common Mistakes:
4. Without a durable message queue like Kafka, what exactly breaks in a chat system's message delivery?
medium
A. The system fails to authenticate users leading to unauthorized access
B. Messages are encrypted incorrectly causing security breaches
C. Messages are duplicated multiple times due to lack of deduplication
D. Alice's send blocks until Bob responds; if Bob's server crashes, message is lost with no retry

Solution

  1. Step 1: Identify role of durable queue

    Kafka decouples sender and receiver; without it, sender waits synchronously for receiver's ack.

  2. Step 2: Trace failure without Kafka

    If Bob's server crashes, no ack arrives, send blocks indefinitely and message is lost with no retry mechanism.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    No Kafka -> synchronous send blocks, message lost if receiver crashes [check]
Hint: No Kafka -> send blocks and message lost if receiver crashes [check]
Common Mistakes:
5. At 10x scale, which component in the URL Shortener system is most likely to become the first bottleneck, and why?
hard
A. The Redirect Service, because it maintains stateful HTTP connections that cannot be easily sharded
B. The Analytics pipeline, because batch processing delays increase with volume
C. The Cache layer, because it runs on a single node and cannot scale horizontally
D. The Database, because it cannot handle the increased number of writes for new short codes

Solution

  1. Step 1: Identify stateful components

    Redirect Service holds stateful HTTP connections (e.g., WebSocket or session affinity), limiting horizontal scaling.

  2. Step 2: Compare scaling challenges

    Database and Cache can be sharded or scaled horizontally; Analytics pipeline can be scaled with more workers; Redirect Service's stateful connections cause scaling bottleneck first.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Stateful Redirect Service limits horizontal scaling, causing first bottleneck [check]
Hint: Stateful services limit scaling before DB or cache do [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard