Block storage vs object storage vs file storage in HLD - Design Approaches Compared

Design: Storage System Comparison

Focus on understanding and designing block storage, object storage, and file storage systems. Out of scope: detailed hardware design or specific vendor implementations.

Functional Requirements

FR1: Support storing and retrieving data efficiently

FR2: Handle different types of data access patterns

FR3: Provide scalability for growing data needs

FR4: Ensure data durability and availability

FR5: Allow easy integration with applications

Non-Functional Requirements

NFR1: Latency for data access should be under 100ms for common operations

NFR2: System should scale to petabytes of data

NFR3: Availability target of 99.9% uptime

NFR4: Support concurrent access by thousands of clients

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Storage nodes or devices

Metadata management service

Access protocols (e.g., NFS, SMB, REST APIs)

Caching layers

Replication and backup mechanisms

Design Patterns

Distributed file system pattern

Key-value store pattern for object storage

Block device abstraction

Caching and tiering strategies

Replication and consistency models

Reference Architecture

Client
  |
  |---> File Storage System (e.g., NFS, SMB)
  |         |
  |         |---> Metadata Server (manages directories, files)
  |         |---> Storage Nodes (store file blocks)
  |
  |---> Object Storage System (e.g., S3)
  |         |
  |         |---> Object API Gateway
  |         |---> Metadata Service (stores object metadata)
  |         |---> Storage Nodes (store objects as blobs)
  |
  |---> Block Storage System (e.g., SAN)
            |
            |---> Block Device Interface
            |---> Storage Arrays (store raw blocks)
            |---> Volume Manager

Components

File Storage System

NFS, SMB, Distributed File Systems

Store and manage files with hierarchical directories and metadata

Object Storage System

REST APIs, S3-compatible systems

Store data as objects with metadata, scalable for unstructured data

Block Storage System

SAN, iSCSI, Fibre Channel

Provide raw block-level storage for low-level data access

Metadata Server

Custom service or database

Manage file or object metadata for quick lookup and organization

Storage Nodes

Disk arrays, SSDs

Physically store data blocks, files, or objects

Access Protocols

NFS, SMB for file; REST for object; iSCSI for block

Enable clients to communicate with storage systems

Request Flow

1. Client sends request to access data.

2. For file storage, client uses file system protocol to request file or directory.

3. Metadata server locates file blocks and returns storage node info.

4. Client reads/writes file blocks from storage nodes.

5. For object storage, client sends REST API request with object key.

6. Object API gateway authenticates and forwards request to metadata service.

7. Metadata service locates object storage node and retrieves or stores object blob.

8. For block storage, client connects to block device interface.

9. Block device maps requests to physical storage arrays.

10. Client reads/writes raw blocks without metadata abstraction.

Database Schema

Entities: - File: id, name, path, size, timestamps, metadata_id - Directory: id, name, parent_directory_id - Object: id, key, size, metadata_id, storage_node_id - Metadata: id, attributes (key-value pairs) - StorageNode: id, type (block/file/object), capacity, status Relationships: - Directory contains Files and other Directories (1:N) - File and Object link to Metadata (1:1) - StorageNode stores Files, Objects, or Blocks (1:N)

Scaling Discussion

Bottlenecks

Metadata server becomes a single point of failure or bottleneck in file and object storage.

Storage nodes can run out of capacity or bandwidth under heavy load.

Network bandwidth limits data transfer speed between clients and storage.

Consistency and synchronization overhead increases with scale.

Solutions

Use distributed metadata services with sharding and replication to avoid bottlenecks.

Add more storage nodes and use load balancing to distribute data and requests.

Implement caching layers near clients to reduce repeated data transfers.

Adopt eventual consistency models where strict consistency is not required to improve performance.

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying use cases, 20 minutes designing and explaining each storage type, 10 minutes discussing scaling and trade-offs, 5 minutes for questions.

Explain differences in data access patterns and use cases for block, file, and object storage.

Describe how metadata management differs and why it matters.

Discuss protocols and client interaction models for each storage type.

Highlight scalability challenges and solutions for large-scale storage systems.

Show understanding of trade-offs between performance, complexity, and flexibility.

Practice

1. Which storage type splits data into fixed-size blocks and provides fast, low-level access?

easy

A. Block storage

B. Object storage

C. File storage

D. Cloud storage

Solution

1. Step 1: Understand block storage concept

   Block storage divides data into blocks and stores them separately for quick access.

2. Step 2: Compare with other storage types

   Object storage stores data as objects with metadata, and file storage organizes data in files and folders.

3. Final Answer:

   Block storage -> Option A

4. Quick Check:

   Block storage = fast block-level access [OK]

Hint: Blocks mean fast raw data access, not files or objects [OK]

Common Mistakes:

• Confusing object storage with block storage
• Thinking file storage splits data into blocks
• Assuming cloud storage is a type here

2. Which of the following correctly describes object storage?

easy

A. Stores data as blocks without metadata

B. Stores data as objects with metadata for easy retrieval

C. Organizes data in hierarchical files and folders

D. Splits data into fixed-size chunks for fast access

Solution

1. Step 1: Recall object storage features

   Object storage saves data as objects, each with metadata describing the data.

2. Step 2: Eliminate incorrect options

   Blocks without metadata describe block storage; hierarchical files describe file storage.

3. Final Answer:

   Stores data as objects with metadata for easy retrieval -> Option B

4. Quick Check:

   Object storage = objects + metadata [OK]

Hint: Objects always have metadata; blocks do not [OK]

Common Mistakes:

• Mixing block storage with object storage
• Thinking file storage uses metadata like objects
• Confusing hierarchical files with object storage

3. A company needs to store large amounts of unstructured data like images and videos with metadata for easy search. Which storage type fits best?

medium

A. Object storage for metadata and scalability

B. Block storage for fast raw access

C. File storage for hierarchical organization

D. Local disk storage for simplicity

Solution

1. Step 1: Analyze data type and requirements

   Unstructured data like images/videos benefit from metadata and scalability.

2. Step 2: Match storage type to needs

   Object storage supports metadata and scales well for large unstructured data sets.

3. Final Answer:

   Object storage for metadata and scalability -> Option A

4. Quick Check:

   Unstructured + metadata = Object storage [OK]

Hint: Metadata + large unstructured data = object storage [OK]

Common Mistakes:

• Choosing block storage for unstructured data
• Assuming file storage handles metadata well
• Picking local disk which lacks scalability

4. A developer tries to use file storage for a distributed app needing fast block-level access but experiences slow performance. What is the likely cause?

medium

A. File storage does not support hierarchical folders

B. Block storage cannot be used in distributed apps

C. File storage is not optimized for block-level raw access

D. Object storage lacks metadata support

Solution

1. Step 1: Understand file storage characteristics

   File storage organizes data in files and folders, optimized for sharing, not raw block access.

2. Step 2: Identify mismatch with app needs

   Fast block-level access requires block storage, not file storage, causing slow performance.

3. Final Answer:

   File storage is not optimized for block-level raw access -> Option C

4. Quick Check:

   File storage ≠ fast block access [OK]

Hint: File storage is for files, not fast block access [OK]

Common Mistakes:

• Believing block storage can't be used distributed
• Confusing object storage metadata with file storage
• Thinking file storage lacks folders

5. You are designing a scalable photo-sharing app. Users upload photos with tags and descriptions. Which storage type should you choose and why?

hard

A. Local disk storage, for simplicity and speed

B. File storage, to organize photos in folders by user

C. Block storage, for fast access to photo bytes

D. Object storage, to store photos with metadata and scale easily

Solution

1. Step 1: Analyze app requirements

   The app needs to store photos with tags/descriptions (metadata) and scale for many users.

2. Step 2: Match storage type to requirements

   Object storage supports metadata per object and scales well for large data and users.

3. Step 3: Evaluate other options

   Block storage is low-level without metadata; file storage is less scalable; local disk lacks scalability and reliability.

4. Final Answer:

   Object storage, to store photos with metadata and scale easily -> Option D

5. Quick Check:

   Photo app + metadata + scale = Object storage [OK]

Hint: Metadata + scale = object storage for apps [OK]

Common Mistakes:

• Choosing block storage ignoring metadata needs
• Assuming file storage scales well for many users
• Picking local disk for a cloud-scale app