Bird
Raised Fist0

Storage Tiering - Hot, Warm, Cold, Archive and Cost Optimization

HLDmediumInterview freq: medium
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/5
Client API

Upload data

Client sends new data to Tiering Controller via HTTP/REST

Protocol:HTTP/REST
BeforeClient has new data ready to store
AfterData received by Tiering Controller for tier assignment
Tiering Controller

Store data in Hot Tier

Tiering Controller writes data to Hot Tier Cassandra cluster

Protocol:Internal RPC
BeforeData is in Tiering Controller memory, not yet persisted
AfterData durably stored in Hot Tier Cassandra
Lifecycle Manager

Move data from Hot to Warm Tier

Lifecycle Manager triggers migration of data older than threshold to Warm Tier

Protocol:Lifecycle events
BeforeData resides in Hot Tier Cassandra
AfterData copied to Warm Tier Cassandra; Hot Tier copy marked for deletion
Lifecycle Manager

Move data from Warm to Cold Tier

Lifecycle Manager migrates older data to Cold Tier (HDFS + HBase) for batch access

Protocol:Lifecycle events
BeforeData present in Warm Tier Cassandra
AfterData stored in Cold Tier HDFS + HBase; Warm Tier copy marked for deletion
Lifecycle Manager

Archive data to Amazon S3 Glacier

Data older than cold tier retention is archived to S3 Glacier for cost savings

Protocol:S3 API
BeforeData stored in Cold Tier HDFS + HBase
AfterData archived in Amazon S3 Glacier; Cold Tier copy marked for deletion
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────┐
│   Client API  │
└───────┬───────┘
        │ HTTP/REST
        ▼
┌───────────────┐
│Tiering Ctrlr  │
└───────┬───────┘
        │ Internal RPC
        ▼
┌───────────────┐
│Hot Tier       │
│- Cassandra    │
└───────┬───────┘
        │ Internal RPC
        ▼
┌───────────────┐
│Warm Tier      │
│- Cassandra    │
└───────┬───────┘
        │ Internal RPC
        ▼
┌───────────────┐
│Cold Tier      │
│- HDFS + HBase │
└───────┬───────┘
        │ Lifecycle events
        ▼
┌───────────────┐
│Lifecycle Mngr │
└───────┬───────┘
        │ S3 API
        ▼
┌───────────────┐
│Archive Tier   │
│- Amazon S3    │
│  Glacier      │
└───────────────┘
Design Rationale

This architecture implements multi-tier storage to optimize cost and performance. Data uploaded via the Client API is first stored in the Hot Tier Cassandra cluster for low-latency access. The Tiering Controller manages initial placement. The Lifecycle Manager automates data migration based on age and access patterns: moving data from Hot to Warm Cassandra, then to Cold Tier HDFS + HBase for batch processing, and finally archiving to Amazon S3 Glacier for long-term cost savings. Each migration step marks the previous tier's copy for deletion, ensuring data is stored only once. This tiered approach balances fast access for recent data with economical storage for older data.

Primary Focus

✓

Practice

(1/5)
1. In a cache system using LFU eviction, at step 4 the eviction process selects a candidate key for removal. What is the state of the candidate's frequency count at this step?
easy
A. The candidate has the highest frequency count among all keys
B. The candidate's frequency count is ignored and eviction is random
C. The candidate's frequency count is reset to zero before eviction
D. The candidate has the lowest frequency count among all keys

Solution

  1. Step 1: Understand LFU eviction criteria

    LFU evicts the cache entry with the lowest access frequency to keep frequently used items.

  2. Step 2: Analyze eviction candidate state at step 4

    The candidate selected for eviction must have the lowest frequency count; frequency is not reset or ignored.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    LFU evicts lowest frequency count keys [check]
Hint: LFU evicts lowest frequency, not highest or random [OK]
Common Mistakes:
2. A system must handle 1 million events per second, ensuring durability and ordered processing per aggregate. Which component best fits this requirement?
easy
A. Use a Redis pub/sub channel for event distribution
B. Use Kafka with partitioning by aggregate ID
C. Use a relational database with triggers for event handling
D. Use RabbitMQ with round-robin queue distribution

Solution

  1. Step 1: Identify Kafka's role in event sourcing

    Kafka supports high throughput with partitioning by aggregate ID, ensuring ordered event processing per aggregate.

  2. Step 2: Evaluate other options

    Redis pub/sub lacks durability guarantees; relational DB triggers don't scale to 1M events/sec; RabbitMQ's round-robin breaks ordering per aggregate.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Kafka partitions by aggregate ID -> ordered, durable event stream [check]
Hint: Kafka partitions preserve order per aggregate [check]
Common Mistakes:
3. If the rollback mechanism is missing in the deployment system, what exact failure occurs when a faulty version is released during Canary deployment?
medium
A. Users experience degraded service indefinitely as faulty version remains live without fallback
B. The previous stable version is restored manually without affecting users
C. The deployment system retries indefinitely, causing deployment pipeline blockage
D. The system automatically disables all new features, causing partial outages

Solution

  1. Step 1: Understand rollback absence impact

    Without rollback, faulty Canary version cannot be reverted automatically.

  2. Step 2: Trace failure effect

    Faulty version remains live, causing degraded user experience indefinitely.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    No rollback -> faulty version stays live -> degraded service persists [check]
Hint: No rollback means faulty version stays live, degrading users indefinitely [OK]
Common Mistakes:
4. In an asynchronous messaging system, what exactly breaks if the message queue component is removed?
medium
A. Senders block until recipients acknowledge, causing high latency
B. Messages are duplicated due to lack of deduplication in the queue
C. Messages are delivered immediately but with no ordering guarantees
D. The system silently drops messages without notifying senders

Solution

  1. Step 1: Understand queue's role in message deduplication and buffering

    Queues often provide deduplication and buffering to ensure exactly-once or at-least-once delivery.

  2. Step 2: Consequence of removing the queue

    Without the queue, messages may be duplicated due to lack of deduplication and buffering mechanisms.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Removing queue causes message duplication due to missing deduplication [check]
Hint: No queue means no deduplication, causing duplicates [check]
Common Mistakes:
5. For a high-throughput e-commerce platform requiring sortable, unique order IDs generated across multiple regions with minimal coordination, which ID generation approach is best?
medium
A. Use Twitter Snowflake-style custom IDs with timestamp and region bits
B. Use UUIDv4 generated randomly at each region
C. Use auto-increment IDs generated by a centralized database
D. Use auto-increment IDs generated independently per region without coordination

Solution

  1. Step 1: Identify requirements

    High throughput, sortable, unique IDs across regions with minimal coordination.

  2. Step 2: Evaluate options

    UUIDv4 (A) is unique but not sortable; centralized auto-increment (C) causes bottlenecks; independent auto-increment (D) risks collisions; Snowflake-style IDs (B) embed timestamp and region, enabling sortable, unique IDs with decentralized generation.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Snowflake IDs -> sortable, unique, decentralized [check]
Hint: Sortable unique IDs at scale -> Snowflake pattern [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard