Bird
Raised Fist0

Twitter Snowflake - Timestamp + Machine ID + Sequence Design

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/6
Configuration Service

Provide machine ID and config

Snowflake node requests its unique machine ID and config parameters

Protocol:Config Fetch
BeforeSnowflake node has no machine ID assigned
AfterSnowflake node has machine ID and config loaded
Snowflake ID Generator Node

Generate unique Snowflake ID

Combines current timestamp, machine ID, and sequence number to create ID

Protocol:Internal
BeforeSnowflake node has machine ID and sequence reset
AfterUnique Snowflake ID generated for new event
Snowflake ID Generator Node

Publish ID event to Kafka

Sends generated Snowflake ID as message to Kafka cluster

Protocol:Kafka Protocol
BeforeSnowflake ID generated but not yet persisted
AfterSnowflake ID durably stored in Kafka log
Kafka Cluster

Stream ID event to Cassandra

Kafka consumer writes Snowflake ID and metadata into Cassandra DB

Protocol:Cassandra CQL
BeforeSnowflake ID message present in Kafka topic
AfterSnowflake ID persisted in Cassandra DB
Cassandra DB

Persist Snowflake ID and metadata

Stores the Snowflake ID and associated event data for durability

Protocol:CQL Write
BeforeSnowflake ID message received from Kafka consumer
AfterSnowflake ID and data durably stored in Cassandra
Monitoring & Alerting System

Collect metrics and alert

Reads ID generation rate and errors from Snowflake node and Kafka

Protocol:REST API
BeforeMetrics not yet collected for this ID generation
AfterMetrics updated and alerts triggered if thresholds exceeded
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────────────────┐
│ Snowflake ID Generator Node   │
│ Generates unique IDs using    │
│ Timestamp + Machine ID + Seq  │
└───────────────┬───────────────┘
                │ Kafka Protocol
                ▼
┌───────────────────────────────┐
│ Kafka Cluster                 │
│ Durable distributed log queue │
└───────────────┬───────────────┘
                │ Cassandra CQL
                ▼
┌───────────────────────────────┐
│ Cassandra DB                  │
│ Stores Snowflake IDs and data │
└───────────────┬───────────────┘
                │ REST API
                ▼
┌───────────────────────────────┐
│ Monitoring & Alerting System  │
│ Tracks ID generation metrics  │
└───────────────┬───────────────┘
                │ Config Fetch
                ▼
┌───────────────────────────────┐
│ Configuration Service         │
│ Provides machine ID and config│
└───────────────────────────────┘
Design Rationale

The Twitter Snowflake system generates unique IDs by combining a timestamp, machine ID, and sequence number within the Snowflake ID Generator Node. This node first obtains its machine ID and configuration from the Configuration Service. Upon generating a unique ID, it publishes the ID event to the Kafka Cluster, which acts as a durable distributed log. Kafka then streams the ID event to Cassandra DB for persistent storage. Meanwhile, the Monitoring & Alerting System collects metrics from the Snowflake node and Kafka to track system health and performance. This architecture ensures scalable, fault-tolerant, and globally unique ID generation with durable persistence and observability.

Primary Focus

✓

Practice

(1/5)
1. A professional networking platform must handle 1 million connection requests per second and ensure eventual consistency without blocking user actions. Which component is best suited to decouple the high write throughput from downstream processing?
easy
A. Use a distributed message queue like Kafka to buffer and asynchronously process connection events
B. Use an in-memory cache like Redis to store connection requests temporarily without persistence
C. Use a relational database with immediate synchronous writes for all connection requests
D. Use a REST API gateway that directly writes to the graph database synchronously

Solution

  1. Step 1: Identify the need for decoupling high write throughput

    Immediate synchronous writes (Use a relational database with immediate synchronous writes for all connection requests and D) block user actions and don't scale well at 1M writes/sec. Redis (Use an in-memory cache like Redis to store connection requests temporarily without persistence) lacks durability and persistence guarantees for critical connection data.

  2. Step 2: Evaluate Kafka's role

    Kafka buffers high write volume asynchronously, ensuring durability and eventual consistency without blocking user requests.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka buffers writes asynchronously -> decouples load [check]
Hint: Use durable async queues to decouple high write load [OK]
Common Mistakes:
2. In a load balancer using Weighted Round Robin, at step 5 the balancer receives a new request. The current server weights are: Server A=5, Server B=3, Server C=2. Which server does the balancer select next?
easy
A. Server A, because Weighted Round Robin cycles through servers proportionally to their weights
B. Server B, because it has fewer active connections than Server A
C. Server C, because it was least recently selected
D. Server A, because it has the highest static weight

Solution

  1. Step 1: Understand Weighted Round Robin operation

    Weighted Round Robin cycles through servers, selecting each server multiple times proportional to its weight before moving to the next.

  2. Step 2: Apply weights to selection

    Server A with weight 5 will be selected more frequently than B or C. At step 5, the balancer continues cycling respecting weights.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Weighted Round Robin respects weights in cycling, not just highest weight always [check]
Hint: Weighted Round Robin cycles servers proportional to weights, not just highest weight [check]
Common Mistakes:
3. For a high-throughput payment processing system, which approach best balances failure isolation and resource utilization?
medium
A. Implement bulkheads with separate thread pools per payment type to isolate failures
B. Use a single large thread pool shared by all payment types to maximize utilization
C. Use circuit breakers on the shared thread pool to isolate failures dynamically
D. Rely on autoscaling the shared thread pool based on load metrics

Solution

  1. Step 1: Evaluate shared thread pool risks

    A single shared pool (A) risks cascading failures; circuit breakers (C) detect failures but don't isolate resource exhaustion. Autoscaling (D) reacts late and may not prevent blast radius.

  2. Step 2: Assess bulkhead benefits

    Bulkheads (B) isolate failures per payment type, preventing one type's overload from affecting others, balancing isolation and utilization.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Bulkheads isolate failures per resource partition [check]
Hint: Bulkheads isolate by partitioning resources per failure domain [check]
Common Mistakes:
4. For the review system, which architecture better supports real-time aggregation of review scores for millions of listings with minimal latency, given that eventual consistency is acceptable?
medium
A. Use a stream processing system like Apache Flink to aggregate reviews in real-time
B. Use a relational database with complex joins and transactions for immediate consistency
C. Use batch processing with daily MapReduce jobs to update aggregated scores
D. Use a NoSQL document store with denormalized review aggregates updated on each write

Solution

  1. Step 1: Identify real-time aggregation needs

    Millions of listings require low-latency updates; immediate consistency is not mandatory.

  2. Step 2: Evaluate architectures

    Relational DB (A) is costly and slow at scale; batch jobs (C) have high latency; NoSQL denormalization (D) risks write amplification and stale data. Stream processing (B) provides scalable, low-latency aggregation with eventual consistency.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Stream processing enables scalable real-time aggregation with eventual consistency [check]
Hint: Stream processing fits real-time aggregation with eventual consistency [check]
Common Mistakes:
5. At 10x scale, which component in a consistent hashing database sharding system is most likely to fail first and why?
hard
A. The central shard metadata service, due to increased coordination overhead
B. The individual database shards, because storage capacity limits are reached first
C. The hash ring management service, because updating and propagating ring changes becomes a bottleneck
D. The client-side routing cache, as it cannot handle the increased number of shards efficiently

Solution

  1. Step 1: Identify scaling bottlenecks

    Central metadata service (The central shard metadata service, due to increased coordination overhead) may be distributed or avoided; shards (The individual database shards, because storage capacity limits are reached first) scale horizontally; client caches (The client-side routing cache, as it cannot handle the increased number of shards efficiently) can be sharded or refreshed incrementally.

  2. Step 2: Analyze hash ring management at scale

    Hash ring management (The hash ring management service, because updating and propagating ring changes becomes a bottleneck) requires updating and propagating ring changes to all nodes; at 10x scale, this coordination overhead and update frequency cause it to become the first bottleneck.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Hash ring updates bottleneck first at large scale [check]
Hint: Hash ring updates bottleneck first at large scale [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard