Bird
Raised Fist0

Time-Series Databases - Metrics, Events, and IoT Patterns

HLDmediumInterview freq: medium
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Time-Series Databases - Metrics, Events, and IoT Patterns
Designing scalable, low-latency time-series databases for metrics, events, and IoT data ingestion and querying
mediumDatabases - NoSQLFreq: medium
⚙️
Functional Requirements
  • Ingest high volume of time-series data points (metrics, events, IoT sensor readings) with timestamps
  • Support efficient range queries over time intervals with filters on tags/labels
  • Implement data retention policies with automatic data expiration and downsampling
  • Provide real-time and batch ingestion pipelines with fault tolerance
  • Support aggregation queries (e.g., avg, max, min) over time windows
  • Allow schema flexibility for heterogeneous IoT device data and dynamic tags
📊
Non-Functional Requirements
  • Handle 2 billion daily active devices generating 40 messages per day each
  • Achieve 99.99% availability for writes and reads
  • Ensure write latency under 50ms P99 and read latency under 100ms P99 for typical queries
  • Support data retention of 1 year with automatic downsampling after 30 days
  • Scale to 925K writes per second peak ingestion rate
❓
Clarifying Questions to Ask
  • What is the expected cardinality of tags/labels per time-series metric?
  • Are writes mostly append-only or do updates/deletes occur frequently?
  • What is the typical query pattern: recent data vs historical data?
  • Is data compression or downsampling required for older data?
  • What is the SLA for data durability and replication?
  • Are there any constraints on data schema evolution or device heterogeneity?
Out of Scope
✗ Complex ad-hoc analytics or OLAP workloads beyond time-series queries
✗ User authentication and authorization mechanisms
✗ Visualization or dashboarding layers
✗ Transactional consistency across multiple time-series
📌
Assumptions
  • 2 billion devices active daily
  • Each device sends 40 messages per day
  • Average message size is 100 bytes
  • Data retention period is 365 days with 3x replication
⚡
QPS Calculation
write qps2B devices × 40 msgs/day ÷ 86400 sec/day ≈ 925,926 writes/sec
peak write qpsAssuming 2× peak factor: 925,926 × 2 ≈ 1.85M writes/sec
read qpsAssuming 10% of write QPS for queries: 925,926 × 0.1 ≈ 92,593 reads/sec
💾
Storage Estimate
daily storage2B × 40 × 100 bytes = 8 TB/day raw data
retention storage8 TB/day × 365 days = 2,920 TB ≈ 2.9 PB raw
with replication2.9 PB × 3 replicas = 8.7 PB total storage
🖥️
Server Estimate
write nodesAssuming each node handles 50K writes/sec, need 1.85M ÷ 50K = 37 nodes
storage nodesAssuming each node stores 100 TB with replication, need 8.7 PB ÷ 100 TB = 87 nodes
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          |  IoT Devices   |
          +-------+--------+
                  | MQTT/HTTP
                  v
          +----------------+
          |  Ingestion API |
          +-------+--------+
                  | Kafka Topics
                  | (partitioned by device_id)
                  v
          +----------------+
          |  Kafka Cluster |
          +-------+--------+
                  | Consumer Groups
                  v
          +----------------+
          |  Write Workers |
          +-------+--------+
                  | Writes
                  v
          +----------------+
          |  Cassandra DB  |
          +-------+--------+
                  | Query API
                  v
          +----------------+
          |  Query Service |
          +----------------+
                  |
                  v
          +-------------------------------+
          | Retention & Downsampling Work |
          +-------------------------------+
                  |
                  v
          +----------------+
          |  Cassandra DB  |
          +----------------+
🧩
Components
Ingestion API
Receives time-series data from devices via MQTT/HTTP, validates and forwards to Kafka
Why thisKafka decouples ingestion from storage, supports high throughput and fault tolerance
Why notDirect writes to Cassandra cause high write latency and backpressure under 1.85M writes/sec
Kafka Cluster
Distributed log for buffering and ordering time-series data, partitioned by device_id
Why thisKafka partitions enable horizontal scaling and ordered processing per device
Why notRabbitMQ lacks Kafka’s throughput and partitioning guarantees at this scale
Write Workers
Consume Kafka topics, batch writes to Cassandra with retries and backpressure handling
Why thisBatching improves Cassandra write efficiency and reduces load spikes
Why notDirect synchronous writes increase latency and reduce throughput
Cassandra DB
Distributed wide-column store optimized for time-series data with tunable consistency
Why thisCassandra’s write-optimized SSTables and time-series data model scale to PBs with low latency
Why notPostgreSQL requires manual sharding and does not scale linearly at 1.85M writes/sec
Query Service
Handles range and aggregation queries over time intervals, interfaces with Cassandra
Why thisCustom query layer enables caching, query optimization, and schema abstraction
Why notDirect client queries to Cassandra expose schema complexity and increase latency
Retention & Downsampling Worker
Periodically downsample old data and delete expired data per retention policy
Why thisSeparates heavy batch processing from real-time ingestion, maintains storage efficiency
Why notTTL-only approach causes unpredictable compaction and storage spikes
🔄
Data Flow
1
IoT Devices → Ingestion API
Send time-series data points via MQTT/HTTP
Devices batch or stream data with timestamps and tags
2
Ingestion API → Kafka Cluster
Publish validated messages to Kafka topic partitioned by device_id
Ensures ordered, durable buffering for downstream processing
3
Kafka Cluster → Write Workers
Consume messages in consumer groups, batch for Cassandra writes
Batching reduces write amplification and improves throughput
4
Write Workers → Cassandra DB
Write time-series data with partition key (device_id) and clustering key (timestamp)
Efficient writes with tunable consistency and compaction
5
Query Service → Cassandra DB
Execute range and aggregation queries over time intervals
Supports low-latency queries with caching and query optimization
6
Retention & Downsampling Worker → Cassandra DB
Delete expired data and downsample older data periodically
Maintains storage limits and improves query performance
🗄️
time_series_data
PK: device_id
device_idtimestampmetric_namevaluetags
Stores raw time-series data points partitioned by device and clustered by timestamp for efficient range queries.
🗄️
downsampled_data
PK: device_id, metric_name
device_idmetric_nametime_window_startavg_valuemax_valuemin_value
Stores aggregated downsampled metrics for older data to reduce storage and speed up queries.
🔌
POST
IngestData
Request
{
  "device_id": "string",
  "timestamp": "long",
  "metrics": "map<string, double>",
  "tags": "map<string, string>"
}
Response
{
  "status": "string",
  "message": "string"
}
Endpoint for devices to send time-series data points in real-time.
🔌
GET
QueryRange
Request
{
  "device_id": "string",
  "metric_name": "string",
  "start_time": "long",
  "end_time": "long",
  "aggregation": "string (optional)"
}
Response
{
  "data_points": "list of {timestamp: long, value: double}",
  "aggregation_result": "map<string, double> (if aggregation requested)"
}
Query raw or aggregated time-series data over a time range.
🔌
POST
ManageRetention
Request
{
  "device_id": "string (optional)",
  "action": "string (e.g., 'downsample', 'delete_expired')"
}
Response
{
  "status": "string",
  "message": "string"
}
Trigger retention policy actions like downsampling or data expiration.

Practice

(1/5)
1. A company needs to store large amounts of unstructured data like images and videos with metadata accessible via HTTP APIs. Which storage type best fits this use case?
easy
A. Object storage, because it stores data as objects with metadata and supports HTTP access.
B. Block storage, because it provides low-latency random access to raw data blocks.
C. File storage, because it offers hierarchical directories and POSIX file system semantics.
D. In-memory storage, because it offers the fastest access to large files.

Solution

  1. Step 1: Identify storage needs for unstructured data with metadata

    Object storage stores data as objects with customizable metadata and supports HTTP APIs, ideal for images/videos.

  2. Step 2: Eliminate other storage types

    Block storage is low-level and lacks metadata support; file storage is optimized for hierarchical file systems but less scalable for massive unstructured data; in-memory storage is volatile and unsuitable for large persistent files.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Object storage = scalable unstructured data + metadata + HTTP [check]
Hint: Object storage = objects + metadata + HTTP access [check]
Common Mistakes:
2. In the video transcoding pipeline, at step 4 the Transcoding Worker checks the job queue for new tasks. What happens if the queue is empty at this step?
easy
A. The Transcoding Worker immediately shuts down to save resources
B. The Transcoding Worker fetches tasks from a backup queue stored in the database
C. The Transcoding Worker polls the queue repeatedly with a short delay until new tasks arrive
D. The Transcoding Worker blocks waiting on a message broker notification until a new task is available

Solution

  1. Step 1: Understand Transcoding Worker queue interaction

    Transcoding Workers typically use message brokers with blocking or long-polling to avoid busy-waiting. Immediate shutdown (A) wastes throughput. Polling with delay (B) causes unnecessary load and latency. Backup queue in DB (D) is inefficient and not standard.

  2. Step 2: Recognize blocking on broker notification

    Blocking on a message broker notification (e.g., Kafka consumer poll or RabbitMQ consumer wait) is efficient and standard to wait for new tasks.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Blocking on broker notification avoids busy polling [check]
Hint: Workers block on broker notification, not busy poll [check]
Common Mistakes:
3. For a large-scale code repository system supporting millions of users, which branching strategy best balances performance and user flexibility?
medium
A. Use a centralized locking mechanism to serialize all branch creation and merges
B. Use lightweight branch pointers referencing immutable commits with eventual consistency
C. Use full branch copies stored separately for each user to avoid conflicts
D. Use synchronous replication of branch metadata across all servers before allowing merges

Solution

  1. Step 1: Analyze branching strategies for scale

    Lightweight branch pointers referencing immutable commits allow fast branch creation and merges without locking, supporting high concurrency.

  2. Step 2: Evaluate other options

    Centralized locking serializes operations, hurting performance; full branch copies waste storage and complicate merges; synchronous replication adds latency and reduces availability.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Immutable commits + lightweight pointers -> scalable branching [check]
Hint: Immutable commits enable lightweight, scalable branches [check]
Common Mistakes:
4. For a distributed cache with frequent node additions and removals, which approach best balances load and minimizes key remapping?
medium
A. Use rendezvous hashing to assign keys to nodes
B. Use a modulo hash on node count with manual rebalancing
C. Use consistent hashing with virtual nodes to smooth load distribution
D. Use a centralized directory to track key-node mappings

Solution

  1. Step 1: Compare consistent hashing and rendezvous hashing

    Both minimize remapping, but consistent hashing with virtual nodes better smooths load across nodes, especially with frequent changes.

  2. Step 2: Evaluate centralized directory and modulo hashing

    Centralized directory adds latency and single point of failure; modulo hashing causes massive remapping and manual rebalancing overhead.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Virtual nodes in consistent hashing smooth load and minimize remapping [check]
Hint: Virtual nodes smooth load in consistent hashing [check]
Common Mistakes:
5. At 10x scale, which component is most likely to fail first in a ChatGPT-like system and why?
hard
A. The LLM inference servers because GPU memory limits prevent scaling beyond a few instances
B. The Chat Servers because WebSocket connections are stateful and cannot be resharded, causing uneven load and connection limits
C. The Kafka cluster because partition count cannot be increased dynamically without downtime
D. The Context Manager because database queries for conversation history become too slow under high load

Solution

  1. Step 1: Analyze stateful components at scale

    Chat Servers maintain WebSocket connections bound to server memory, limiting horizontal scaling and resharding.

  2. Step 2: Compare other components' scalability

    LLM servers (A) can be scaled with model parallelism; Kafka (C) supports partition increases with planning; Context Manager DB (D) can be sharded or cached. Stateful WebSocket binding (B) is the hardest bottleneck.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Stateful WebSocket binding limits Chat Server scaling first [check]
Hint: Stateful WebSockets block scaling; stateless components scale easier [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard