Bird
Raised Fist0

Stateless vs Stateful Services - Design Implications at Scale

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Stateless vs Stateful Services - Design Implications at Scale
Understanding trade-offs between stateless and stateful services for scalable distributed systems
mediumCore Distributed Systems ConceptsFreq: high
Prerequisites:System Design FundamentalsCAP Theorem
⚙️
Functional Requirements
  • Support user requests with minimal latency under high load (2B DAU scale)
  • Maintain session or user-specific state where necessary
  • Enable horizontal scaling of services without downtime
  • Provide fault tolerance and graceful recovery from failures
  • Support consistent data access patterns for stateful components
  • Allow easy deployment and rollback of service versions
📊
Non-Functional Requirements
  • Handle peak write QPS of approximately 925K writes/sec
  • Achieve 99.99% availability SLA for user-facing services
  • Ensure 100ms P99 latency for read and write operations
  • Support data retention of 30 days with replication factor 3
  • Maintain operational simplicity to reduce MTTR (Mean Time To Recovery)
❓
Clarifying Questions to Ask
  • What is the expected read-to-write ratio for the service?
  • Are there any strict consistency requirements or eventual consistency acceptable?
  • What is the expected session duration or state lifetime for stateful services?
  • Is the state shared across multiple instances or isolated per instance?
  • What failure scenarios are most critical to handle (e.g., node crash, network partition)?
  • Are there any existing infrastructure constraints or preferred technologies?
Out of Scope
✗ Designing specific database schemas for application data
✗ Authentication and authorization mechanisms
✗ Detailed network security and encryption protocols
✗ Client-side caching strategies
📌
Assumptions
  • 2 billion daily active users (DAU)
  • Each user generates 40 messages or requests per day
  • Average message/request size is 100 bytes
  • Data retention period is 30 days with replication factor 3
⚡
QPS Calculation
write qps2,000,000,000 × 40 / 86,400 ≈ 925,926 writes/sec
peak write qpsAssuming 2× peak factor: 925,926 × 2 ≈ 1,851,852 writes/sec
read qpsAssuming 10 reads per write: 925,926 × 10 ≈ 9,259,260 reads/sec
💾
Storage Estimate
daily storage gb(2,000,000,000 × 40 × 100) / (1024^3) ≈ 7.45 TB/day
retention storage gb7.45 TB × 30 days ≈ 223.5 TB
with replication gb223.5 TB × 3 replicas ≈ 670.5 TB
🖥️
Server Estimate
write pathAssuming each write server handles 10K writes/sec → 1,851,852 / 10,000 ≈ 186 servers
read pathAssuming each read server handles 50K reads/sec → 9,259,260 / 50,000 ≈ 186 servers
🏗️
Architecture Diagram
Architecture Diagram
          +-------------+
          |   Clients   |
          +------+------+
                 |
         +-------v--------+
         |  Load Balancer |
         +-------+--------+
                 |
      +----------+-----------+
      |                      |
 +----v----+           +-----v-----+
 |Stateless|           | Stateful  |
 | Service |           |  Service  |
 +----+----+           +-----+-----+
      |                      |
 +----v----+           +-----v-----+
 | Kafka   |           | Cassandra |
 | Cluster |           |  Cluster  |
 +----+----+           +-----+-----+
      |                      |
 +----v----+           +-----v-----+
 | Redis   |           | Zookeeper |
 | Cache   |           |  Cluster  |
 +---------+           +-----------+
🧩
Components
Load Balancer
Distributes incoming client requests evenly across stateless and stateful service instances
Why thisNginx or Envoy provides high throughput and supports health checks for failover
Why notHardware load balancers add cost and reduce deployment agility
Stateless Service
Handles requests that do not require session affinity or local state, enabling easy horizontal scaling
Why thisKubernetes pods running stateless microservices allow fast scaling and rolling upgrades
Why notStateful services here would limit scaling and complicate deployment
Stateful Service
Manages user session state or long-lived connections requiring affinity and local state
Why thisStatefulSet in Kubernetes with persistent volumes supports stable identity and storage
Why notStateless services cannot maintain session state without external storage, increasing latency
Kafka Cluster
Acts as a durable, distributed commit log for asynchronous event processing and decoupling services
Why thisKafka scales to millions of messages per second with partitioning and replication
Why notRabbitMQ lacks Kafka’s throughput and partitioning scalability at 1.8M writes/sec
Cassandra Cluster
Stores stateful data with high write throughput and tunable consistency across multiple data centers
Why thisCassandra’s masterless architecture avoids single points of failure and supports linear write scaling
Why notPostgreSQL requires complex manual sharding and does not scale linearly at 1.8M writes/sec
Redis Cache
Provides low-latency caching for frequently accessed state or session data to reduce DB load
Why thisRedis TTL natively expires stale data without cron jobs - sub-millisecond lookup
Why notMemcached lacks persistence and advanced eviction policies needed for session data
Zookeeper Cluster
Coordinates distributed stateful services, manages leader election and configuration metadata
Why thisZookeeper provides strong consistency and watches for coordination in Cassandra and StatefulSets
Why notEtcd has higher latency under heavy load and less mature integration with Cassandra
🔄
Data Flow
1
Clients → Load Balancer
Client sends request
Load balancer routes based on request type (stateless vs stateful)
2
Load Balancer → Stateless Service
Forward stateless requests
Stateless services handle requests without session affinity
3
Load Balancer → Stateful Service
Forward stateful requests with session affinity
Stateful services maintain local session state
4
Stateless Service → Kafka Cluster
Publish events asynchronously
Decouples processing and enables eventual consistency
5
Stateful Service → Cassandra Cluster
Read/write session or user state
Strong write throughput with tunable consistency
6
Stateless/Stateful Service → Redis Cache
Cache hot session or user data
Reduces latency and database load for frequently accessed data

Practice

(1/5)
1. In a system implementing causal consistency, at step 4 the Read Coordinator receives a read request for key K after a write W causally related to a previous write V. What state does the Read Coordinator ensure before returning the value for K?
easy
A. It returns the latest value of K without waiting, since eventual consistency suffices
B. It waits until all writes causally preceding W, including V, are visible to the reader
C. It returns the value from the local replica immediately, ignoring causal dependencies
D. It synchronizes with the leader to enforce linearizability before responding

Solution

  1. Step 1: Trace Read Coordinator behavior at Step 4

    Under causal consistency, the Read Coordinator must ensure that all causally prior writes (like V) are visible before returning W's value to maintain causal order.

  2. Step 2: Eliminate other options

    It returns the latest value of K without waiting, since eventual consistency suffices ignores causal dependencies; It returns the value from the local replica immediately, ignoring causal dependencies breaks causal guarantees by returning stale data; It synchronizes with the leader to enforce linearizability before responding enforces stronger linearizability, not required here.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Read Coordinator waits for causal dependencies before returning [check]
Hint: Causal consistency waits for all prior causally related writes [check]
Common Mistakes:
2. In a messaging system, at step 4 the Delivery Worker checks the message queue for undelivered messages. What happens if the message's delivery TTL has expired at this step?
easy
A. The Delivery Worker marks the message as expired and removes it from the queue
B. The Delivery Worker discards the message without retrying or marking it
C. The Delivery Worker moves the message to a dead-letter queue but keeps it in the main queue
D. The Delivery Worker retries sending the message indefinitely until success

Solution

  1. Step 1: Understand Delivery Worker behavior at Step 4

    It checks message TTL to decide if delivery attempts should continue.

  2. Step 2: TTL expiration handling

    Expired messages are discarded without retry or marking to avoid resource waste; indefinite retries (A) cause resource exhaustion; marking expired (B) is not standard; dead-letter queue (C) is for failed, not expired messages.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Expired messages are discarded to maintain throughput and latency balance [check]
Hint: Expired messages get discarded immediately, no retries [check]
Common Mistakes:
3. In a system where Step 4 is the Delivery Worker reading from a Kafka topic partitioned by user ID, what is the Delivery Worker's state if it reads a message but the downstream notification service is temporarily unavailable?
easy
A. The message is marked as delivered and removed from Kafka
B. The Delivery Worker skips the message and moves to the next
C. The message is lost because Kafka does not guarantee delivery
D. The message remains uncommitted in Kafka and will be retried

Solution

  1. Step 1: Understand Kafka consumer commit semantics

    Kafka commits offsets only after successful processing. If downstream is unavailable, the message is not committed.

  2. Step 2: Delivery Worker retries uncommitted messages

    Uncommitted messages remain in the topic and will be redelivered on retry, ensuring no message loss.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Kafka offset commit after processing -> retries on failure [check]
Hint: Kafka commits offsets only after successful processing [OK]
Common Mistakes:
4. You must choose between a monolithic architecture and microservices for a system with rapidly evolving features and a small team. Which choice best justifies your design decision to the interviewer?
medium
A. Choose microservices because they always scale better regardless of team size
B. Choose monolith to reduce operational complexity and speed up feature delivery for a small team
C. Choose microservices to isolate failures even if it increases deployment overhead
D. Choose monolith because it requires no testing or CI/CD pipelines

Solution

  1. Step 1: Analyze team size and feature velocity constraints

    Small teams benefit from simpler architectures to reduce overhead and speed delivery.

  2. Step 2: Evaluate operational complexity of microservices

    Microservices add deployment and operational complexity, which can slow small teams.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Monolith reduces complexity, speeds delivery for small teams [check]
Hint: Small teams prefer monoliths for simplicity and speed [check]
Common Mistakes:
5. At 10x scale, which component in a push-based newsfeed system is most likely to fail first and why?
hard
A. The fan-out service, because pushing to millions of followers causes write amplification bottleneck
B. The database, because it cannot handle 10x writes per second
C. The cache layer, because it runs out of memory storing all user feeds
D. The web servers, because they cannot handle 10x concurrent HTTP connections

Solution

  1. Step 1: Analyze database scaling

    Databases can be sharded and scaled horizontally; not usually the first bottleneck.

  2. Step 2: Analyze fan-out service scaling

    Push-based fan-out causes write amplification proportional to follower count; at 10x scale, this bottleneck grows superlinearly and saturates first.

  3. Step 3: Analyze cache and web servers

    Caches can be scaled with eviction and sharding; web servers scale with load balancers and stateless design.

  4. Final Answer:

    Option A -> Option A

  5. Quick Check:

    Fan-out write amplification causes first bottleneck at 10x scale [check]
Hint: Fan-out write amplification hits first at 10x scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard