Bird
Raised Fist0

Availability - Active-Active vs Active-Passive Failover Patterns

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
🎯
Availability - Active-Active vs Active-Passive Failover Patterns
45 min · senior
senior

You are designing the availability strategy for a 50 million daily active user (DAU) social media platform that requires end-to-end encryption, offline message delivery, and a P99 latency under 200ms globally. The system must handle regional datacenter failures gracefully. Your task is to design and compare Active-Active and Active-Passive failover patterns to meet these constraints.

  • End-to-end encryption must be preserved across failovers
  • Offline message delivery with guaranteed eventual consistency
  • P99 request latency under 200ms globally
  • Support for multi-region deployment with automatic failover
  • Data consistency trade-offs must be explicit and justified
📋
Guided Steps
1/5
Step 1
Choosing the Availability Pattern
Which availability pattern-Active-Active or Active-Passive-best suits the 50M DAU system with P99 < 200ms and why?
💡 Hint
Consider trade-offs between latency, consistency, and complexity in multi-region deployments.
Use Active-Active with multi-region Apache Cassandra clusters configured with lightweight transactions for conflict resolution to achieve low latency and high availability.
Mid: Names Active-Active for availability. Senior: Explains Cassandra's tunable consistency and lightweight transactions to handle conflicts and maintain P99 latency under 200ms globally.
⚠️Common Wrong Answer
Choosing Active-Passive without considering failover latency leads to >1s downtime and violates P99 latency SLA.
Step 2
Data Replication and Consistency Model
How will you replicate data across regions and what consistency model will you enforce to balance availability and correctness?
💡 Hint
Think about Cassandra's replication factor and consistency levels like QUORUM or LOCAL_QUORUM.
Configure Cassandra with NetworkTopologyStrategy replication and LOCAL_QUORUM reads/writes per region to minimize cross-region latency while ensuring strong-enough consistency.
Mid: Mentions replication factor and eventual consistency. Senior: Details LOCAL_QUORUM to reduce latency and explains trade-offs with cross-region consistency and failover scenarios.
⚠️Common Wrong Answer
Using ALL consistency level causes high latency and impacts P99 SLA under load.
Step 3
Failover Detection and Traffic Routing
How will the system detect failures and route traffic during failover to maintain availability and encryption guarantees?
💡 Hint
Consider DNS-based routing with health checks and session affinity mechanisms.
Use AWS Route 53 with health checks and latency-based routing combined with Envoy sidecars for session affinity and TLS termination to preserve encryption end-to-end.
Mid: Describes DNS failover. Senior: Explains Route 53 health checks, latency-based routing, and Envoy sidecars preserving TLS and session affinity during failover.
⚠️Common Wrong Answer
Relying solely on DNS TTL changes causes slow failover and breaks encryption guarantees.
Step 4
Offline Message Delivery and Synchronization
How will offline message delivery and synchronization be handled to ensure eventual consistency and minimal user impact?
💡 Hint
Think about client-side caching, message queues, and conflict resolution strategies.
Implement client-side encrypted message queues with Apache Kafka topics partitioned by conversation_id, combined with Cassandra for durable storage and conflict resolution via vector clocks.
Mid: Mentions message queues for offline delivery. Senior: Explains Kafka partitioning by conversation_id for ordering, vector clocks for conflict resolution, and Cassandra for durable storage.
⚠️Common Wrong Answer
Using simple HTTP polling for offline sync causes excessive load and inconsistent message ordering.
Step 5
API Design for Availability and Failover
Design 3 core API endpoints that support sending, syncing, and acknowledging messages under the availability constraints.
💡 Hint
How does the client send encrypted messages ensuring delivery during failover?How does the client synchronize offline messages after reconnecting?How does the recipient acknowledge message receipt to maintain consistency?
Define POST /messages/send with idempotency keys, GET /messages/sync with cursor-based pagination, and POST /messages/ack with message IDs to update delivery status in Cassandra.
Mid: Names endpoints and basic functionality. Senior: Details idempotency, cursor-based sync for efficient offline recovery, and acknowledgement to update delivery state atomically.
⚠️Common Wrong Answer
Designing endpoints without idempotency or cursor-based sync leads to duplicated messages and inefficient offline recovery.
🔌
API Design Challenge
Sketch 3 core API endpoints to support availability and failover patterns in this system.
  • How does the client send encrypted messages ensuring delivery during failover?
  • How does the client synchronize offline messages after reconnecting?
  • How does the recipient acknowledge message receipt to maintain consistency?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Clear choice and justification of Active-Active vs Active-Passive pattern with scale reasoning
  • Explicit replication and consistency model using real tech like Cassandra with LOCAL_QUORUM
  • Failover detection and routing design with AWS Route 53 and Envoy preserving encryption
  • Offline delivery mechanism using Kafka partitioning and conflict resolution
  • API design with idempotency, cursor-based sync, and acknowledgement endpoints
Good to Have - 30%
  • Discussion of trade-offs between latency and consistency in multi-region setup
  • Explanation of vector clocks or lightweight transactions for conflict resolution
  • Consideration of session affinity and TLS termination in failover routing
  • Handling of client-side caching and offline message queueing
Bonus - 10%
  • Inclusion of metrics and monitoring strategies for failover detection
  • Design for scaling Kafka partitions and Cassandra nodes with 50M DAU
  • Security considerations for key management in end-to-end encryption
Using ALL consistency level in Cassandra causes high latency and SLA violations, leading to P99 latency exceeding 200ms under load
Failover relying solely on DNS TTL changes causes slow recovery and downtime, breaking encryption guarantees due to session loss
HTTP polling at 50M DAU results in 50 million reconnects per minute just to check for messages, collapsing under load and causing inconsistent message ordering
✅
Model Answer
▼ Reveal

For a 50M DAU social media platform requiring end-to-end encryption, offline message delivery, and P99 latency under 200ms globally, an Active-Active multi-region architecture using Apache Cassandra and Kafka is optimal. This design balances availability, latency, and consistency while preserving encryption and supporting offline scenarios.

  • Use Active-Active pattern with Cassandra lightweight transactions to handle conflicts and maintain low latency
  • Configure Cassandra with LOCAL_QUORUM consistency per region to minimize cross-region latency while ensuring strong consistency
  • Implement failover detection using AWS Route 53 health checks combined with Envoy sidecars to preserve TLS and session affinity
  • Use Kafka partitioned by conversation_id to guarantee message ordering and vector clocks for conflict resolution
  • Design APIs with idempotency keys for sending messages, cursor-based pagination for syncing offline messages, and acknowledgement endpoints to update delivery status atomically
  • Active-Active multi-region Apache Cassandra with NetworkTopologyStrategy for replication to ensure availability and low latency
  • Use Cassandra LOCAL_QUORUM consistency level to balance latency and data correctness across regions
  • AWS Route 53 latency-based routing with health checks for automatic failover and minimal downtime
  • Envoy sidecars for session affinity and TLS termination preserving end-to-end encryption during failover
  • Apache Kafka partitioned by conversation_id for ordered offline message queuing and delivery
  • Vector clocks for conflict resolution in Active-Active writes to maintain eventual consistency
  • Client-side encrypted message queues and cursor-based sync APIs to support offline delivery and recovery
  • API endpoints with idempotency keys and acknowledgement mechanisms to prevent duplicates and ensure delivery state consistency

Practice

(1/5)
1. A web application needs to ensure that cache entries are updated immediately after every write to the database to prevent stale reads. Which caching pattern best fits this requirement?
easy
A. Cache Aside, where the application updates the cache only on cache misses
B. Write-Through, where writes go through the cache and update the database synchronously
C. Read-Through, where the cache loads data on reads but writes bypass the cache
D. Write-Behind, where writes are buffered and asynchronously flushed to the database

Solution

  1. Step 1: Identify cache update timing

    Write-Through updates cache and database synchronously, ensuring cache is always fresh.
  2. Step 2: Eliminate other patterns

    Cache Aside updates cache only on misses, risking stale cache after writes; Read-Through only loads on reads; Write-Behind delays DB update, risking stale cache.
  3. Final Answer:

    Option B -> Option B
  4. Quick Check:

    Write-Through synchronously updates cache and DB [check]
Hint: Write-Through syncs cache and DB on writes [check]
Common Mistakes:
2. In the video transcoding pipeline, at step 4 the Transcoding Worker checks the job queue for new tasks. What happens if the queue is empty at this step?
easy
A. The Transcoding Worker immediately shuts down to save resources
B. The Transcoding Worker fetches tasks from a backup queue stored in the database
C. The Transcoding Worker polls the queue repeatedly with a short delay until new tasks arrive
D. The Transcoding Worker blocks waiting on a message broker notification until a new task is available

Solution

  1. Step 1: Understand Transcoding Worker queue interaction

    Transcoding Workers typically use message brokers with blocking or long-polling to avoid busy-waiting. Immediate shutdown (A) wastes throughput. Polling with delay (B) causes unnecessary load and latency. Backup queue in DB (D) is inefficient and not standard.
  2. Step 2: Recognize blocking on broker notification

    Blocking on a message broker notification (e.g., Kafka consumer poll or RabbitMQ consumer wait) is efficient and standard to wait for new tasks.
  3. Final Answer:

    Option D -> Option D
  4. Quick Check:

    Blocking on broker notification avoids busy polling [check]
Hint: Workers block on broker notification, not busy poll [check]
Common Mistakes:
3. A distributed system requires a lock mechanism that ensures mutual exclusion across multiple Redis instances with minimal risk of deadlocks. Which component or approach best handles this requirement?
easy
A. Implement Redlock algorithm coordinating locks across multiple Redis masters
B. Use a relational database row lock with distributed transactions
C. Use a single Redis instance with SETNX and expire for locking
D. Use a centralized ZooKeeper lock service for distributed locking

Solution

  1. Step 1: Understand distributed lock requirements

    Single Redis instance with SETNX risks single point of failure and split-brain issues.
  2. Step 2: Evaluate Redlock's multi-instance coordination

    Redlock uses multiple Redis masters to reduce risk of deadlocks and ensure safety under partial failures.
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    Redlock coordinates multiple Redis masters for safer distributed locks [check]
Hint: Multiple Redis masters coordinated -> safer distributed locks [check]
Common Mistakes:
4. You need to model a social media feed with millions of users and high write throughput. Which data modeling approach is best to optimize for write scalability and read performance?
medium
A. Embed all posts and comments inside each user document to minimize joins.
B. Use a graph database to model user relationships and posts for efficient traversal.
C. Store all posts in a single collection with a compound index on user and timestamp for fast queries.
D. Use a referencing pattern with separate collections for users, posts, and comments, and denormalize popular posts into user feed documents.

Solution

  1. Step 1: Analyze write scalability and read patterns

    High write throughput and millions of users require scalable writes and efficient feed reads.
  2. Step 2: Evaluate referencing with denormalization

    Referencing separates concerns; denormalizing popular posts into feeds optimizes read performance without large documents.
  3. Final Answer:

    Option D -> Option D
  4. Quick Check:

    Referencing plus denormalization balances write scale and read speed [check]
Hint: Reference plus denormalize popular data for scalable writes and fast reads [check]
Common Mistakes:
5. At 10x scale, which component is most likely to fail first due to caching, and why?
medium
A. The database, because it cannot handle the increased write load from cache invalidations
B. The network, because cache synchronization traffic overwhelms bandwidth
C. The cache, because it runs out of memory due to increased data volume
D. The application servers, because cache misses cause excessive DB queries

Solution

  1. Step 1: Identify scaling impact on cache

    At 10x scale, cache memory pressure increases due to more data and entries.
  2. Step 2: Evaluate failure likelihood

    Cache memory exhaustion is common before DB or network fail due to caching.
  3. Final Answer:

    Option C -> Option C
  4. Quick Check:

    Cache memory limits cause failure first at scale [check]
Hint: Cache memory exhaustion causes failure first at scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard