Bird
Raised Fist0

Redis-Based Distributed Lock - Redlock Algorithm Design

HLDmediumInterview freq: medium
Choose your preparation mode4 modes available
🏗️
Redis-Based Distributed Lock - Redlock Algorithm Design
Design a fault-tolerant distributed lock service using Redis and the Redlock algorithm
mediumDistributed CoordinationFreq: medium
Prerequisites:Caching BasicsKey-Value StoresDistributed Systems FundamentalsConsensus and Coordination Algorithms
⚙️
Functional Requirements
  • Provide mutual exclusion lock acquisition and release across distributed clients
  • Ensure lock safety: no two clients hold the same lock simultaneously
  • Support automatic lock expiration to avoid deadlocks (TTL-based)
  • Allow clients to attempt lock acquisition with timeout and retry logic
  • Support lock renewal (extend TTL) by the lock owner
  • Provide a simple API for lock operations (acquire, release, renew)
📊
Non-Functional Requirements
  • High availability: 99.99% uptime for lock operations
  • Low latency: lock acquire/release P99 latency under 50ms
  • Scalability: support up to 1 million lock requests per second globally
  • Fault tolerance: tolerate up to 2 Redis node failures without violating safety
  • Consistency: ensure strong mutual exclusion despite network partitions
❓
Clarifying Questions to Ask
  • What is the expected number of concurrent clients and locks?
  • What is the maximum acceptable lock TTL and renewal frequency?
  • Are locks expected to be short-lived or long-lived on average?
  • Is the system deployed across multiple data centers or a single region?
  • What failure scenarios must the system tolerate (network partitions, Redis crashes)?
  • Should the system support reentrant locks or only simple exclusive locks?
Out of Scope
✗ Distributed consensus algorithms like Paxos or Raft implementation
✗ Lock fairness or priority scheduling guarantees
✗ Multi-resource or transactional locking semantics
✗ Client-side retry/backoff policies beyond basic timeout
📌
Assumptions
  • 2 billion daily active users globally
  • Each user performs 40 lock operations per day on average
  • Average lock operation payload size is 100 bytes
  • Retention period for logs/metrics is 7 days
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 writes/sec
peak write qpsAssuming 3× peak factor: 925,926 × 3 ≈ 2.78M writes/sec
read qpsAssuming 1.5× writes for reads (lock status checks): 925,926 × 1.5 ≈ 1.39M reads/sec
💾
Storage Estimate
daily storage2B × 40 × 100 bytes = 8 TB/day
retention storage8 TB × 7 days = 56 TB
with replication56 TB × 3 (replication factor) = 168 TB total
🖥️
Server Estimate
redis cluster nodesAssuming 50k writes/sec per Redis node, need 2.78M / 50k ≈ 56 nodes
client api serversAssuming 10k QPS per API server, need (2.78M + 1.39M) / 10k ≈ 427 servers
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          | Client Service |
          +-------+--------+
                  |
                  | Lock API Calls (Acquire/Release/Renew)
                  |
          +-------v--------+
          | API Gateway    |
          +-------+--------+
                  |
                  |
    +-------------+-------------+
    |             |             |
+---v---+     +---v---+     +---v---+
|Redis 1|     |Redis 2|     |Redis 5|
+-------+     +-------+     +-------+
    |             |             |
    +-------------+-------------+
                  |
          +-------v--------+
          | Lock Manager   |
          +----------------+
🧩
Components
Client Service
Initiates lock operations (acquire, release, renew) from distributed clients
Why thisClients interact with the lock system via a simple API to coordinate distributed tasks using Redis
Why notDirect Redis access by clients is rejected due to security and complexity concerns with Redis cluster exposure
API Gateway
Receives client lock requests, performs authentication, rate limiting, and forwards to Lock Manager
Why thisCentralizes client request handling and protects Redis cluster from direct exposure using API Gateway technology
Why notBypassing API Gateway risks uncontrolled load and security vulnerabilities that Redis alone cannot mitigate
Redis Cluster (5 independent nodes)
Stores lock keys with TTLs; each node independently manages a subset of locks
Why thisRedis supports fast in-memory TTL expiration and atomic commands (SET NX PX) essential for Redlock algorithm
Why notMemcached lacks atomic TTL support; Zookeeper is heavier and slower for high QPS distributed locking
Lock Manager
Implements Redlock algorithm logic: attempts lock acquisition on majority of Redis nodes
Why thisCentralizes distributed lock coordination ensuring safety and fault tolerance per Redlock design using Redis commands
Why notClient-side Redlock logic is error-prone and complicates failure handling without centralized coordination
Monitoring & Metrics
Tracks lock success/failure rates, latency, Redis node health, and alerts on anomalies
Why thisEssential for operational visibility and proactive failure detection in distributed locking using monitoring tools like Prometheus
Why notNo monitoring risks undetected failures and SLA violations impacting system reliability
🔄
Data Flow
1
Client Service → API Gateway
Send lock acquire request with lock key and TTL
Client initiates lock acquisition with timeout and unique client ID
2
API Gateway → Lock Manager
Forward validated lock request
API Gateway authenticates and rate-limits before passing request
3
Lock Manager → Redis Cluster nodes
Attempt SET NX PX command on each Redis node in parallel
Lock Manager tries to acquire lock on majority (≥3 of 5) nodes
4
Redis Cluster nodes → Lock Manager
Return success/failure for lock acquisition
Each node responds atomically if lock was acquired or not
5
Lock Manager → API Gateway
Aggregate responses and decide lock success
Lock succeeds only if majority nodes granted lock within TTL
6
API Gateway → Client Service
Return lock acquisition result
Client receives success or failure with lock token
7
Client Service → API Gateway
Send lock release or renew request with token
Client releases or extends lock TTL before expiration
8
Lock Manager → Redis Cluster nodes
Execute DEL or SET PX commands atomically with token verification
Ensures only lock owner can release or renew lock
🗄️
locks
PK: lock_key
lock_key (string)owner_id (string)expiration_time (timestamp)token (string)
Stores lock state with owner and expiration for distributed locks.
🗄️
lock_metrics
PK: date
date (date)lock_key (string)acquire_count (int)release_count (int)failure_count (int)
Aggregates lock operation metrics for monitoring and alerting.
🔌
Acquire Lock
Request
{
  "lock_key": "string",
  "ttl_ms": "int",
  "client_id": "string"
}
Response
{
  "success": "boolean",
  "token": "string (present if success)"
}
Attempts to acquire a lock with specified TTL; returns token if successful.
🔌
Release Lock
Request
{
  "lock_key": "string",
  "token": "string"
}
Response
{
  "success": "boolean"
}
Releases the lock if token matches the owner.
🔌
Renew Lock
Request
{
  "lock_key": "string",
  "token": "string",
  "ttl_ms": "int"
}
Response
{
  "success": "boolean"
}
Extends the lock TTL if token matches the owner.

Practice

(1/5)
1. In a primary-replica replication system, at step 4 the replica receives a batch of write-ahead log entries from the primary. What is the replica's exact state after applying these entries?
easy
A. Replica marks entries as committed and acknowledges primary before applying
B. Replica applies entries to its data store and then acknowledges primary
C. Replica applies entries and immediately serves reads from them before acknowledgment
D. Replica buffers entries without applying until primary confirms commit

Solution

  1. Step 1: Trace replica behavior at step 4

    Replica applies WAL entries to its local store to catch up with primary.
  2. Step 2: Understand acknowledgment timing

    Replica acknowledges primary only after applying entries to ensure durability and consistency.
  3. Final Answer:

    Option B -> Option B
  4. Quick Check:

    Replica applies then acknowledges -> ensures data durability [check]
Hint: Apply WAL entries before ack -> durability ensured [check]
Common Mistakes:
2. A search engine must quickly update document scores as new documents arrive and queries are frequent. Which component is best suited to efficiently compute term frequency and inverse document frequency in real-time?
easy
A. An in-memory inverted index with incremental TF-IDF updates
B. A relational database with full-text indexing
C. A batch processing system that recomputes TF-IDF nightly
D. A simple key-value store without indexing capabilities

Solution

  1. Step 1: Identify real-time scoring needs

    Relational DBs with full-text indexing (A) are slower for frequent updates; batch recomputation (C) is too delayed for real-time queries.
  2. Step 2: Evaluate indexing and update efficiency

    In-memory inverted index with incremental TF-IDF updates (B) supports fast updates and queries; key-value stores (D) lack indexing for relevance scoring.
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    In-memory inverted index enables fast incremental TF-IDF updates [check]
Hint: In-memory inverted index enables fast TF-IDF updates [check]
Common Mistakes:
3. For detecting anomalies in high-volume metric streams with low latency, which architecture is better suited and why?
medium
A. Batch processing with hourly aggregation to reduce noise and improve accuracy
B. Polling external ML service every 10 minutes to classify anomalies asynchronously
C. Streaming anomaly detection using windowed statistical models for near real-time alerts
D. Threshold-based alerts combined with manual tuning to approximate anomalies

Solution

  1. Step 1: Analyze latency and volume constraints

    High-volume streams require low-latency detection to be actionable.
  2. Step 2: Match architecture to constraints

    Streaming anomaly detection with windowed models processes data continuously, enabling near real-time alerts, unlike batch or polling which add latency.
  3. Final Answer:

    Option C -> Option C
  4. Quick Check:

    Streaming windowed models -> low latency anomaly detection [check]
Hint: High volume + low latency -> streaming anomaly detection [check]
Common Mistakes:
4. For a social network with users having highly variable follower counts (from tens to millions), which fan-out architecture best balances write latency and read latency?
medium
A. Pure pull fan-out: followers fetch posts on demand from a global store
B. Pure push fan-out: push every post to all followers' feeds synchronously
C. Hybrid fan-out: push to high-follower-count users' feeds, pull for low-follower-count users
D. Push fan-out only for active users, pull for inactive users

Solution

  1. Step 1: Analyze pure push fan-out (A)

    Push to all followers synchronously causes huge write amplification for celebrities, increasing write latency and storage.
  2. Step 2: Analyze pure pull fan-out (C)

    Pull reduces write cost but increases read latency and complexity for users with many followers.
  3. Step 3: Evaluate hybrid fan-out (B)

    Hybrid pushes to users with many followers (to reduce read latency) and pulls for users with few followers (to reduce write amplification), balancing both latencies.
  4. Step 4: Evaluate push only for active users (D)

    Activity-based push doesn't address follower count skew effectively.
  5. Final Answer:

    Option C -> Option C
  6. Quick Check:

    Hybrid fan-out balances write and read latency by follower count [check]
Hint: Hybrid fan-out balances write/read latency by follower count [check]
Common Mistakes:
5. For storing aggregated logs with fast query response and cost efficiency, which storage architecture is best when logs are mostly write-once and rarely updated?
medium
A. Use a distributed relational database with strong consistency
B. Use a NoSQL key-value store with frequent compactions
C. Use a columnar data warehouse optimized for append-only writes
D. Use a distributed file system with manual indexing

Solution

  1. Step 1: Analyze log storage requirements

    Logs are write-once, large volume, and require fast analytical queries.
  2. Step 2: Evaluate columnar data warehouse

    Columnar stores optimize for append-only writes and fast scans, reducing cost and improving query speed.
  3. Final Answer:

    Option C -> Option C
  4. Quick Check:

    Columnar warehouse -> optimized for append-only and analytics [check]
Hint: Columnar storage excels at append-only analytics [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard