Bird
Raised Fist0

ULID and KSUID - Sortable Distributed IDs for Modern Systems

HLDmediumInterview freq: medium
Choose your preparation mode4 modes available
🎯
ULID and KSUID - Sortable Distributed IDs for Modern Systems
45 min · senior
senior

You are building a distributed ID generation service for a 50 million daily active user (DAU) startup. The system must generate globally unique, time-sortable IDs that can be generated offline and online, support encryption for privacy compliance, and guarantee P99 latency under 200ms. These IDs will be used as primary keys in multiple downstream systems including databases and messaging queues. The system must handle bursts of up to 10 million ID requests per second globally.

  • IDs must be lexicographically sortable by generation time to enable efficient range queries.
  • Encryption of IDs is required to prevent leakage of timestamp or machine identifiers.
  • Offline ID generation must be supported for edge devices with intermittent connectivity.
  • P99 latency for ID generation must be under 200ms globally.
  • The system must handle up to 10 million ID generation requests per second.
  • IDs must be collision-resistant across multiple data centers without coordination.
📋
Guided Steps
1/5
Step 1
Choosing the ID Format and Encoding
What ID format and encoding scheme will you choose to ensure global uniqueness, time-sortability, and encryption compatibility?
💡 Hint
Consider ULID and KSUID formats and how their timestamp and randomness components are structured and encoded.
Use ULID (128-bit) or KSUID (160-bit) format with Crockford's Base32 encoding for lexicographic sorting and compact representation; this supports time-sortability and offline generation.
Mid: Names ULID or KSUID and mentions time-sortability. Senior: Explains timestamp bit allocation, randomness entropy, and why Base32 encoding enables lexicographic sorting and easy integration with encryption.
⚠️Common Wrong Answer
Using UUIDv4 which is random and not time-sortable, causing inefficient range queries and poor indexing performance.
Step 2
Ensuring Collision Resistance at Scale
How will you guarantee collision resistance across multiple data centers and offline clients without coordination?
💡 Hint
Think about entropy sources, machine identifiers, and how ULID/KSUID embed randomness and timestamp to avoid collisions.
Embed a high-entropy cryptographically secure random component combined with a millisecond timestamp and optionally a machine ID; reject using centralized coordination or database-generated IDs due to latency and scalability limits.
Mid: Mentions randomness plus timestamp. Senior: Details how ULID uses 80 bits randomness and KSUID uses 128 bits randomness, and why adding machine ID or cryptographically secure RNG prevents collisions at 10M QPS globally.
⚠️Common Wrong Answer
Using a centralized ID generator or database auto-increment causing bottlenecks and single points of failure.
Step 3
Supporting Encryption and Privacy
How will you encrypt IDs to prevent leaking timestamp or machine info while preserving sortability and uniqueness?
💡 Hint
Consider format-preserving encryption or deterministic encryption schemes compatible with lexicographic order.
Use Format-Preserving Encryption (FPE) such as NIST-approved FF1 mode with AES-256 to encrypt the ULID/KSUID while preserving lexicographic order; reject naive encryption that breaks sort order or increases ID length.
Mid: Suggests encrypting IDs. Senior: Explains why FPE is required to preserve lexicographic order and how AES-FF1 mode works; discusses trade-offs in encryption overhead and key management.
⚠️Common Wrong Answer
Encrypting IDs with standard AES-CBC or AES-GCM which breaks sort order and makes range queries impossible.
Step 4
API Design for Online and Offline ID Generation
Design 3 core API endpoints that support online ID generation, offline ID validation, and encrypted ID decryption.
💡 Hint
Think about endpoints for requesting new IDs, validating offline-generated IDs, and decrypting IDs for downstream systems.
Define endpoints: (1) POST /generate for online encrypted ID generation with millisecond timestamp; (2) POST /validate for offline clients to validate ID format and uniqueness; (3) POST /decrypt to extract timestamp and randomness securely from encrypted IDs using AES-FF1 keys.
Mid: Names endpoints and basic functionality. Senior: Details request/response payloads, latency targets, encryption key rotation, and offline validation logic.
⚠️Common Wrong Answer
Designing only synchronous online generation endpoint ignoring offline use cases, causing offline clients to fail.
Step 5
Scaling the ID Generation Service
How will you architect the system to handle 10 million ID requests per second globally with P99 latency under 200ms?
💡 Hint
Consider stateless services, load balancing, caching, and offline generation to reduce load on central services.
Deploy stateless ID generation microservices behind global load balancers (e.g., AWS ALB or GCP LB), use cryptographically secure RNG libraries locally, and allow offline generation with periodic sync; reject stateful coordination or database lookups for ID generation due to latency and throughput bottlenecks.
Mid: Mentions horizontal scaling and stateless services. Senior: Explains RNG seeding, entropy pool management, offline generation fallback, and how to measure and maintain P99 latency under 200ms globally.
⚠️Common Wrong Answer
Using a single centralized service or database for ID generation causing latency spikes and outages at scale.
🔌
API Design Challenge
Sketch 3 core API endpoints for the ID generation service that support online and offline generation, encryption, and validation.
  • How does a client request a new ID from the online service?
  • How does an offline client generate and later validate an ID?
  • How does the system verify and decrypt an ID to extract timestamp and uniqueness info?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Use ULID or KSUID with Base32 encoding for time-sortable IDs
  • Embed cryptographically secure randomness plus timestamp for collision resistance
  • Use Format-Preserving Encryption (AES-FF1) to preserve sort order and privacy
  • Design stateless scalable microservices for ID generation
  • Define API endpoints supporting online generation, offline validation, and decryption
Good to Have - 30%
  • Support offline ID generation with embedded timestamp and randomness
  • Implement key management with rotation and secure storage
  • Measure and maintain P99 latency under 200ms globally
Bonus - 10%
  • Discuss trade-offs in encryption overhead and key management
  • Explain RNG seeding and entropy pool management
  • Detail offline generation fallback mechanisms
✅
Model Answer
▼ Reveal

The system uses ULID or KSUID formats with Base32 encoding to generate lexicographically sortable, globally unique IDs. It embeds a millisecond timestamp and a cryptographically secure random component to ensure collision resistance at 10 million QPS globally. Format-Preserving Encryption (AES-FF1) encrypts IDs to preserve sort order and privacy. Stateless microservices deployed globally behind load balancers handle online generation with P99 latency under 200ms. Offline generation is supported by embedding timestamp and randomness without coordination. The API exposes /generate, /validate, and /decrypt endpoints. A key management system handles encryption keys with rotation and secure storage.

  • Use ULID (128-bit) or KSUID (160-bit) with Crockford's Base32 encoding for lexicographic sorting and offline generation
  • Embed 80 bits (ULID) or 128 bits (KSUID) of cryptographically secure randomness combined with millisecond timestamp and optional machine ID to avoid collisions without coordination
  • Encrypt IDs using NIST-approved Format-Preserving Encryption (AES-FF1 mode with AES-256) to preserve lexicographic order and privacy compliance
  • Deploy stateless ID generation microservices globally behind load balancers to handle 10M QPS with P99 latency under 200ms
  • Support offline ID generation by embedding timestamp and randomness locally without coordination, with later validation via /validate endpoint
  • Define API endpoints: POST /generate for online encrypted ID generation, POST /validate for offline ID validation, POST /decrypt for extracting timestamp and randomness securely
  • Implement key management with secure storage and rotation policies to maintain encryption security
  • ULID or KSUID format with Base32 encoding for lexicographic time-sortable IDs supporting offline generation
  • Cryptographically secure random component combined with millisecond timestamp to ensure collision resistance at 10M QPS globally
  • Format-Preserving Encryption (AES-FF1) to encrypt IDs preserving lexicographic order and enabling privacy compliance
  • Stateless microservices deployed globally behind load balancers for horizontal scaling and low latency
  • API endpoints: /generate for online encrypted ID generation, /validate for offline ID validation, /decrypt for extracting timestamp and randomness
  • Offline ID generation supported by embedding timestamp and randomness without coordination
  • Key management system for encryption keys with rotation and secure storage
  • Reject centralized coordination or database-generated IDs due to scalability and latency constraints

Practice

(1/5)
1. A distributed key-value store must guarantee that once a client writes a value, any subsequent read by the same client returns that value or a newer one, even if the client switches servers. Which consistency model best supports this requirement?
easy
A. Read-your-writes consistency, because it guarantees the client sees its own writes regardless of server
B. Causal consistency, because it orders all operations globally
C. Eventual consistency, because the system will converge to the latest value eventually
D. Strong consistency, because it ensures all clients see the latest write immediately

Solution

  1. Step 1: Understand client-centric guarantees

    Read-your-writes consistency ensures that a client always sees its own writes, even if it switches servers, without requiring global synchronization.
  2. Step 2: Eliminate other models

    Strong consistency enforces global order but is costly and not required here; eventual consistency does not guarantee immediate visibility of own writes; causal consistency orders related operations but does not guarantee read-your-writes for a single client.
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    Read-your-writes consistency ensures client sees own writes immediately [check]
Hint: Read-your-writes means client always sees own writes [check]
Common Mistakes:
2. A system needs to serve AI code completions with sub-100ms latency while ensuring up-to-date code context. Which component is best suited to provide fast, low-latency retrieval of relevant code snippets?
easy
A. A relational database with complex SQL queries for code indexing
B. An in-memory vector search engine with approximate nearest neighbor (ANN) indexing
C. A distributed file system storing raw code files for batch processing
D. A message queue to buffer code snippet requests before processing

Solution

  1. Step 1: Identify low-latency retrieval need

    Relational DBs (A) have high query latency for vector similarity; distributed file systems (C) are for storage, not fast retrieval; message queues (D) buffer requests but don't serve retrieval.
  2. Step 2: Recognize vector search role

    ANN-based in-memory vector search engines (B) provide fast approximate nearest neighbor lookups, ideal for sub-100ms latency code snippet retrieval.
  3. Final Answer:

    Option B -> Option B
  4. Quick Check:

    ANN vector search enables fast similarity queries [check]
Hint: Vector search engines excel at fast similarity retrieval [check]
Common Mistakes:
3. In a system design interview, at step 4, you explain your choice of caching layer to reduce latency. What happens at this step when the cache misses the requested data?
easy
A. The cache silently returns stale data without fetching fresh data
B. The cache duplicates the request to multiple backends simultaneously
C. The cache returns an error immediately to the client without fallback
D. The cache triggers a backend fetch and updates itself before returning data

Solution

  1. Step 1: Trace cache behavior on miss at step 4

    On a cache miss, the cache should fetch fresh data from the backend to serve the request.
  2. Step 2: Confirm cache updates itself before responding

    This ensures subsequent requests hit the cache, reducing latency and load on backend.
  3. Final Answer:

    Option D -> Option D
  4. Quick Check:

    Cache miss triggers backend fetch and cache update [check]
Hint: Cache miss triggers backend fetch and update [check]
Common Mistakes:
4. In a storage tiering system, at Step 4 the Data Migration Worker checks the access frequency metadata for a data block. What action does it take if the access frequency falls below the warm tier threshold?
easy
A. Immediately deletes the data block to free space
B. Promotes the data block to hot storage for faster access
C. Moves the data block from warm storage to cold storage tier
D. Archives the data block to tape storage without metadata update

Solution

  1. Step 1: Understand migration triggers

    Data Migration Worker uses access frequency to decide tier movement; low frequency triggers move to colder tier.
  2. Step 2: Identify correct tier movement

    Data with low access frequency moves from warm to cold storage, not deletion or promotion.
  3. Final Answer:

    Option C -> Option C
  4. Quick Check:

    Low access frequency -> warm to cold migration [check]
Hint: Low access frequency -> move down tiers, not delete [check]
Common Mistakes:
5. If the distributed lock service is unavailable, what exactly breaks in the hotel booking system's reservation flow?
medium
A. Reservations proceed normally but double bookings increase due to lack of coordination
B. Reservation requests block indefinitely waiting for locks, causing user timeouts and failed bookings
C. Inventory service marks rooms as available incorrectly, causing UI to show wrong availability
D. Payment processing fails because it depends on the lock service for transaction ordering

Solution

  1. Step 1: Identify lock service role in reservation flow

    The lock service ensures exclusive access to room inventory during reservation to prevent double booking.
  2. Step 2: Trace failure impact

    Without the lock service, Inventory cannot reliably mark rooms as unavailable, leading to stale or incorrect availability states shown to users (Inventory service marks rooms as available incorrectly, causing UI to show wrong availability). Reservations may still proceed but with inconsistent inventory state. Blocking indefinitely (Reservation requests block indefinitely waiting for locks, causing user timeouts and failed bookings) is unlikely as fallback mechanisms exist. Payment processing (Payment processing fails because it depends on the lock service for transaction ordering) is independent of locking. Double bookings (Reservations proceed normally but double bookings increase due to lack of coordination) may increase but the immediate failure is incorrect inventory state.
  3. Final Answer:

    Option C -> Option C
  4. Quick Check:

    Lock service failure causes stale inventory availability state [check]
Hint: Lock service failure -> stale inventory state, not immediate booking block [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard