Bird
Raised Fist0

Consistent Hashing for Database Sharding at Scale

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/6
Client

Sends read request

Client issues a read request for user data with a specific key

Protocol:HTTP Request
BeforeClient has user key and wants data
AfterRequest sent to Consistent Hash Ring Service
Consistent Hash Ring Service

Maps key to shard node

Uses consistent hashing to find responsible shard node for key

Protocol:RPC (gRPC)
BeforeRequest received with user key
AfterShard node address identified and request forwarded
Lookup Service (Redis)

Checks cache for shard location

Performs Redis GET to find cached shard node location

Protocol:Redis GET
BeforeShard node address requested by hash ring service
AfterShard node location returned from cache
Shard Nodes (Cassandra)

Processes read query

Shard node executes query and reads data locally

Protocol:Internal Query
BeforeQuery received for user data
AfterData retrieved from local storage
Replication Manager

Ensures data consistency

Replicates data changes to other nodes asynchronously

Protocol:Internal Replication
BeforeData read completed on primary shard
AfterReplication state updated for consistency
Monitoring & Metrics (Prom + Graf)

Collects metrics

Receives metrics push about request latency and errors

Protocol:Metrics Push
BeforeRequest processed by shard and replication manager
AfterMetrics stored and visualized for monitoring
🏗️
Architecture Diagram
Architecture Diagram
┌─────────┐
│ Client  │
└─────┬───┘
      │ HTTP Request
      ▼
┌───────────────────────────────┐
│ Consistent Hash Ring Service   │
│ (maps key to shard node)       │
└─────────────┬─────────────────┘
              │ RPC (gRPC)
              ▼
┌───────────────────────────────┐
│ Lookup Service (Redis)         │
│ (cache shard location)         │
└─────────────┬─────────────────┘
              │ Redis GET
              ▼
┌───────────────────────────────┐
│ Shard Nodes (Cassandra)        │
│ (store user data, handle query)│
└─────────────┬─────────────────┘
              │ Internal Replication
              ▼
┌───────────────────────────────┐
│ Replication Manager            │
│ (ensures data consistency)    │
└─────────────┬─────────────────┘
              │ Metrics Push
              ▼
┌───────────────────────────────┐
│ Monitoring & Metrics (Prom + Graf) │
│ (Prometheus + Grafana)         │
└───────────────────────────────┘
Design Rationale

This architecture uses a Consistent Hash Ring Service to map data keys to specific shard nodes in Apache Cassandra, enabling scalable database sharding. The Lookup Service (Redis Cluster) caches shard locations to reduce lookup latency. When a client sends a read request, it is routed through the hash ring to the correct shard node, which processes the query locally. The Replication Manager asynchronously ensures data consistency across replicas. Monitoring & Metrics components collect operational data for system health visualization. This design balances fast lookups, scalability, and consistency by combining caching, consistent hashing, and replication.

Primary Focus

✓

Practice

(1/5)
1. A global CDN operator needs to prevent a misconfiguration from propagating incorrect BGP routes that could cause widespread internet outages. Which component is primarily responsible for detecting and containing such route leaks?
easy
A. Using a centralized DNS firewall to block suspicious IP prefixes
B. Implementing BGP route filters and prefix limits on edge routers
C. Relying on client-side validation of IP addresses before routing
D. Deploying a distributed cache to store valid routing tables

Solution

  1. Step 1: Identify BGP route filtering role

    BGP route filters and prefix limits on edge routers prevent invalid or excessive route announcements, containing route leaks.

  2. Step 2: Eliminate other options

    DNS firewalls (A) do not control BGP routing; client-side validation (C) is ineffective for routing control; distributed caches (D) do not enforce routing policies.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    BGP route filters contain route leaks at the network edge [check]
Hint: Edge routers enforce BGP filters to block bad routes [check]
Common Mistakes:
2. A web API must limit incoming requests to 100 per second with bursts allowed up to 200 requests. Which component best enforces this rate limiting with burst handling?
easy
A. A fixed window counter that resets every second
B. A simple request queue that drops requests when full
C. A leaky bucket algorithm that processes requests at a fixed rate without bursts
D. A token bucket algorithm that refills tokens at a steady rate with a maximum bucket size

Solution

  1. Step 1: Identify rate limiting with burst support

    The token bucket algorithm allows tokens to accumulate up to a max bucket size, enabling bursts beyond the steady refill rate.

  2. Step 2: Eliminate other options

    Fixed window counters cause spikes at window boundaries and don't handle bursts well. Leaky bucket smooths traffic but disallows bursts. Simple queues drop excess requests without rate control.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Token bucket allows bursts by token accumulation [check]
Hint: Token bucket = steady refill + burst capacity [OK]
Common Mistakes:
3. If the durability mechanism (write-ahead log) is disabled in a transactional database, what exact failure occurs when the system crashes during a commit?
medium
A. The transaction partially commits, causing inconsistent database state
B. The database blocks all new transactions until recovery completes
C. The transaction appears committed to the client but is lost after crash, causing data loss
D. The transaction is rolled back automatically, preserving consistency

Solution

  1. Step 1: Role of write-ahead log (WAL)

    WAL ensures durability by recording changes before commit; without it, recovery cannot guarantee committed data.

  2. Step 2: Trace failure on crash without WAL

    Database must block new transactions and perform full recovery to avoid inconsistent state, causing availability loss.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    No WAL -> recovery blocks new transactions to maintain consistency [check]
Hint: No WAL means recovery blocks new transactions [check]
Common Mistakes:
4. For a CI/CD system that must support both rapid iterative deployments and guaranteed rollback safety, which deployment strategy is most appropriate?
medium
A. Blue-Green Deployment, because it minimizes downtime by switching traffic instantly
B. Rolling Deployment, because it updates servers one by one without extra infrastructure
C. Recreate Deployment, because it fully replaces old versions ensuring clean state
D. Canary Deployment, because it allows gradual exposure and easy rollback on failure

Solution

  1. Step 1: Analyze rollback safety and rapid iteration needs

    Canary deployment gradually exposes new versions, enabling quick rollback if issues arise, balancing speed and safety.

  2. Step 2: Evaluate other strategies

    Blue-Green minimizes downtime but requires double infrastructure; Rolling updates lack gradual exposure and rollback granularity; Recreate causes downtime and no gradual rollback.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Canary balances rapid deployment with rollback safety [check]
Hint: Canary = gradual rollout + rollback safety [check]
Common Mistakes:
5. If Kafka is unavailable, what exactly breaks in the tweet delivery pipeline?
medium
A. Tweets are stored in the DB but timeline caches become permanently stale
B. Tweets are lost immediately with no retry and followers see no updates
C. Tweet ingestion blocks; fan-out delivery waits synchronously causing user write latency spikes
D. Delivery workers switch to synchronous RPC calls causing partial delivery and blocking

Solution

  1. Step 1: Role of Kafka in delivery pipeline

    Kafka buffers tweets asynchronously for fan-out; unavailability stops new fan-out events.

  2. Step 2: Trace failure impact

    Tweets still persist in DB, but without Kafka, timeline caches don't get updated, causing stale timelines.

  3. Step 3: Eliminate other options

    Immediate loss (A) is prevented by DB persistence; ingestion blocking (B) unlikely as writes decouple; synchronous RPC fallback (D) not implemented.

  4. Final Answer:

    Option A -> Option A

  5. Quick Check:

    Kafka outage stalls fan-out, causing stale timeline caches [check]
Hint: Kafka outage stalls fan-out, stale timelines appear [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard