Bird
Raised Fist0

Ticket Server Pattern for Centralized ID Generation

HLDmediumInterview freq: medium
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Ticket Server Pattern for Centralized ID Generation
Centralized ticket server to generate unique, scalable, and ordered IDs
mediumUnique ID GenerationFreq: medium
Prerequisites:Distributed Systems BasicsCAP TheoremConsensus ProtocolsDatabase Partitioning and Replication
⚙️
Functional Requirements
  • Generate globally unique IDs on request with no collisions
  • Support monotonic increasing or time-ordered IDs for sorting
  • Provide IDs with low latency (sub-10ms per request)
  • Allow batch ID allocation to reduce network overhead
  • Expose APIs for ID generation to multiple distributed clients
  • Support horizontal scaling and failover of the ticket server
📊
Non-Functional Requirements
  • Handle peak write QPS of at least 1 million ID requests per second
  • Ensure 99.99% availability with automatic failover
  • P99 latency for ID generation under 10ms
  • Durability of issued IDs with no loss or duplication
  • Support retention of issued ID metadata for 30 days for audit
❓
Clarifying Questions to Ask
  • What is the expected ID format? Numeric, string, or composite?
  • Is strict global ordering required or eventual ordering acceptable?
  • Are IDs required to be k-ordered or strictly monotonic?
  • What is the expected peak and average QPS for ID generation?
  • Is batching of IDs per request allowed or only single ID per call?
  • What failure modes must the system tolerate (network, node crash)?
Out of Scope
✗ Designing client-side ID generation or UUID schemes
✗ Global clock synchronization or external time services
✗ Downstream usage of IDs (e.g., database sharding or indexing)
✗ Complex multi-region consensus protocols like Paxos or Raft
📌
Assumptions
  • 2 billion daily active users (DAU)
  • Each user generates 40 ID requests per day on average
  • Each ID request payload is approximately 100 bytes
  • Retention period for issued ID metadata is 30 days
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 writes/sec
peak write qpsAssuming 3× peak factor: 925,926 × 3 ≈ 2.78M writes/sec
read qpsAssuming 10% read rate for ID metadata: 925,926 × 0.1 ≈ 92,593 reads/sec
💾
Storage Estimate
daily storage bytes2B × 40 × 100 bytes = 8 TB/day
retention storage bytes8 TB × 30 days = 240 TB
with replicationAssuming 3× replication: 240 TB × 3 = 720 TB total
🖥️
Server Estimate
ticket serversAssuming each server handles 100K writes/sec, need 28 servers for peak load
metadata storage nodesUsing Cassandra nodes with 5 TB/node capacity, need 48 nodes for 240 TB data
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          | Client Services|
          +--------+-------+
                   |
                   v
          +--------+--------+
          |  Load Balancer  |
          +--------+--------+
                   |
        +----------+-----------+
        |                      |
 +-------v-------+      +-------v-------+
 | Ticket Server |      | Ticket Server |
 +-------+-------+      +-------+-------+
        |                      |
        +----------+-----------+
                   |
          +--------v--------+
          |  Cassandra DB   |
          +-----------------+
🧩
Components
Load Balancer
Distributes incoming ID generation requests evenly across ticket servers
Why thisAWS ALB or NGINX provides high throughput and health checks for failover
Why notDNS round-robin lacks health awareness and can cause uneven load
Ticket Server
Generates unique IDs using a centralized ticket allocation algorithm
Why thisCustom service with in-memory counters and batch allocation reduces latency and contention
Why notZooKeeper sequential nodes cause high latency and do not scale beyond 100K QPS
Cassandra DB
Stores issued ID metadata for durability and audit with high write throughput
Why thisCassandra supports linear write scaling, tunable consistency, and 3x replication
Why notPostgreSQL requires manual sharding and does not scale linearly at 2M writes/sec
Cache Layer (Redis)
Caches recent ticket allocations and ID ranges to reduce DB reads
Why thisRedis TTL natively expires stale cache entries with sub-millisecond lookup
Why notMemcached lacks persistence and TTL management, risking stale data
Monitoring & Alerting
Tracks latency, error rates, and server health for proactive failure detection
Why thisPrometheus + Grafana provide real-time metrics and alerting with minimal overhead
Why notCustom logging lacks real-time alerting and is harder to scale
🔄
Data Flow
1
Client Services → Load Balancer
Send ID generation request
Clients request unique IDs via REST/gRPC API
2
Load Balancer → Ticket Server
Forward request to least loaded ticket server
Load balancer distributes requests evenly
3
Ticket Server → Cache Layer
Check cache for available ID batch
If cached batch available, allocate ID locally
4
Ticket Server → Cassandra DB
If cache empty, allocate new ID batch and persist
Persist batch allocation to ensure durability
5
Ticket Server → Cache Layer
Update cache with new ID batch
Cache updated to serve subsequent requests quickly
6
Ticket Server → Client Services
Return unique ID(s) to client
Response includes allocated unique ID(s)
7
Monitoring & Alerting → Operations Team
Send alerts on latency or error threshold breaches
Ensures system health and quick incident response

Practice

(1/5)
1. In a system using asynchronous replication, at step 4 the Replication Worker reads the write-ahead log entry for a committed transaction. What is the state of the replica after this step if the worker crashes before applying the entry?
easy
A. Replica has applied the transaction but not acknowledged it
B. Replica has not applied the transaction and will retry on restart
C. Replica has partially applied the transaction causing inconsistency
D. Replica has acknowledged the transaction despite crash

Solution

  1. Step 1: Trace Replication Worker reading WAL entry

    Worker reads the log but crashes before applying, so replica state unchanged.

  2. Step 2: Understand retry behavior on restart

    Since transaction not applied, worker retries applying on restart ensuring consistency.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Unapplied transaction retried after crash [check]
Hint: Unapplied logs retried after crash, no partial state [check]
Common Mistakes:
2. A webhook delivery system must guarantee ordered delivery of events per subscriber, even under retries and failures. Which component is best suited to enforce per-subscriber ordering?
easy
A. A partitioned message queue like Kafka keyed by subscriber ID
B. A distributed cache like Redis to store last delivered event per subscriber
C. A relational database with a global sequence number column
D. A load balancer that routes events to the same delivery worker

Solution

  1. Step 1: Identify ordering guarantee mechanism

    Kafka partitions keyed by subscriber ID ensure all events for that subscriber go to the same partition, preserving order.

  2. Step 2: Eliminate other options

    Redis cache lacks strong ordering guarantees under retries; relational DB global sequence causes contention and latency; load balancer routing alone doesn't guarantee ordering without stateful processing.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka partition key -> per-subscriber ordering guaranteed [check]
Hint: Partition by subscriber ID for ordering [check]
Common Mistakes:
3. If the cache warming service crashes after partially warming keys but before marking completion, what is the immediate failure observed during launch?
medium
A. The cache contains stale data from previous runs causing incorrect responses
B. The cache warming service automatically retries from the start, causing duplicate warming and wasted resources
C. The system falls back to a read-through cache mode with no performance impact
D. Clients experience cache misses for keys not warmed, causing increased latency and possible DB overload

Solution

  1. Step 1: Trace failure impact of partial warming crash

    Without completion mark, warming restarts causing duplicate warming and resource waste.

  2. Step 2: Eliminate other options

    Stale data unlikely if warming overwrites; cache misses occur but duplicate warming is immediate failure; fallback read-through causes latency, not immediate failure.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Crash without completion causes duplicate warming retries [check]
Hint: Crash without completion mark causes duplicate warming retries [check]
Common Mistakes:
4. If the health monitoring component of the GSLB fails silently, what is the most likely failure observed by end users?
medium
A. Users are routed to unhealthy data centers causing request failures or timeouts
B. Traffic is evenly distributed regardless of data center health, causing overload
C. DNS queries fail to resolve, causing complete service outage
D. Users experience increased latency due to rerouting through backup data centers

Solution

  1. Step 1: Understand health monitor role

    The health monitor informs GSLB which data centers are healthy; without it, GSLB cannot avoid unhealthy endpoints.

  2. Step 2: Analyze failure impact

    Without health info, GSLB routes users to unhealthy data centers causing failures (Users are routed to unhealthy data centers causing request failures or timeouts). Increased latency (Users experience increased latency due to rerouting through backup data centers) or DNS failures (DNS queries fail to resolve, causing complete service outage) are indirect or unlikely. Even traffic distribution (Traffic is evenly distributed regardless of data center health, causing overload) ignores health but is less precise than routing to unhealthy nodes.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Silent health monitor failure -> routing to unhealthy data centers [check]
Hint: No health data -> users sent to unhealthy data centers [check]
Common Mistakes:
5. At 10x scale, which component in a multi-tier storage system is most likely to become the bottleneck first, and why?
hard
A. Archive storage because tape drives have limited concurrent access bandwidth
B. Metadata service because it maintains global access frequency and tiering decisions, causing high coordination overhead
C. Cold storage because object storage scales linearly with data size
D. Warm storage because HDDs have slower seek times but can be scaled horizontally easily

Solution

  1. Step 1: Identify component roles at scale

    Archive storage uses tape drives with limited concurrent access bandwidth and high latency.

  2. Step 2: Analyze scaling challenges

    At 10x scale, tape drives become bottleneck due to limited parallelism and slow seek times, unlike metadata which can be sharded.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Archive tape bandwidth limits -> first bottleneck [check]
Hint: Tape archive bandwidth limits scale poorly [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard