NoSQL database types (document, key-value, column, graph) in HLD - Scalability & System Analysis

Scalability Analysis - NoSQL database types (document, key-value, column, graph)

Growth Table: NoSQL Database Types Scaling

Users / Data Size	Key-Value Store	Document Store	Column Store	Graph Database
100 users	Single server, in-memory caching	Single server, simple queries	Single server, small tables	Single server, small graph
10K users	Sharding by key, cache layer	Replica sets, indexing	Partitioning columns, compression	Indexing nodes, caching paths
1M users	Multiple shards, distributed cache	Horizontal scaling, query optimization	Distributed storage, column pruning	Graph partitioning, query parallelism
100M users	Global sharding, multi-region clusters	Multi-region replication, CDN for static data	Massive parallel processing, tiered storage	Advanced graph partitioning, caching hot subgraphs

First Bottleneck

At small scale, the database server CPU and memory limits are the first bottleneck for all NoSQL types because they handle all queries and data in one place.

As users grow to 10K-1M, network bandwidth and disk I/O become bottlenecks due to increased data movement and query complexity.

For graph databases, complex traversals cause CPU bottlenecks earlier than others because graph queries are compute-intensive.

Scaling Solutions

• Key-Value Stores: Use sharding by key, add distributed caching (e.g., Redis), and replicate data for availability.
• Document Stores: Implement replica sets for fault tolerance, add indexes on query fields, and shard collections by document attributes.
• Column Stores: Partition data by columns, compress data to save space, and use distributed storage systems.
• Graph Databases: Partition graphs into subgraphs, cache frequently accessed paths, and parallelize graph queries.
• Across all types, use load balancers to distribute traffic and CDNs to serve static content.

Back-of-Envelope Cost Analysis

• At 1M users, assuming 1 request per second per user, total requests = 1 million QPS.
• Single server handles ~5000 QPS, so need ~200 servers for database layer.
• Storage: If average document/record size is 1 KB, 1M users with 100 records each = 100 GB data.
• Network bandwidth: 1M QPS * 1 KB = ~1 GB/s (8 Gbps), requiring multiple network interfaces and data centers.

Interview Tip

Start by explaining the NoSQL type and its data model. Then discuss expected load and data size. Identify the first bottleneck logically (usually database CPU or disk). Propose scaling solutions matching the bottleneck. Mention trade-offs like consistency vs availability. Use real numbers to show understanding.

Self Check

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add read replicas and implement caching to reduce load on the primary database. If write load grows, shard data horizontally to distribute writes.

Key Result

NoSQL databases scale by sharding and replication; the first bottleneck is usually database CPU and disk I/O, solved by horizontal scaling and caching.

Practice

1. Which NoSQL database type stores data as flexible JSON-like documents, allowing nested structures?

easy

A. Document database

B. Key-value database

C. Column database

D. Graph database

Solution

1. Step 1: Understand document database structure

   Document databases store data in JSON-like documents that can have nested fields and arrays.

2. Step 2: Compare with other NoSQL types

   Key-value stores use simple key and value pairs, column stores organize data by columns, and graph databases focus on relationships.

3. Final Answer:

   Document database -> Option A

4. Quick Check:

   Nested JSON-like data = Document database [OK]

Hint: Documents = JSON-like nested data [OK]

Common Mistakes:

• Confusing key-value stores with document stores
• Thinking column stores store JSON documents
• Mixing graph databases with document databases

2. Which of the following is the correct way to describe a key-value store?

easy

A. Stores data as interconnected nodes and edges

B. Stores data in tables with rows and columns

C. Stores data as simple pairs of keys and values

D. Stores data as nested JSON documents

Solution

1. Step 1: Define key-value store

   Key-value stores save data as pairs: a unique key and its associated value, like a dictionary.

2. Step 2: Eliminate other options

   Graphs use nodes and edges, column stores use tables with columns, and document stores use nested JSON.

3. Final Answer:

   Stores data as simple pairs of keys and values -> Option C

4. Quick Check:

   Key-value = key and value pairs [OK]

Hint: Key-value = simple pairs [OK]

Common Mistakes:

• Confusing graph databases with key-value stores
• Thinking key-value stores use tables
• Mixing document stores with key-value stores

3. Given a graph database storing people and their friendships, which query result would you expect from a query asking for all friends of 'Alice'?

medium

A. A list of key-value pairs with friend names

B. A set of nodes connected to 'Alice' by edges labeled 'friend'

C. A table with columns for friend names and ages

D. A JSON document with nested friend details

Solution

1. Step 1: Understand graph database query output

   Graph queries return nodes and edges; friends of 'Alice' are nodes connected by 'friend' edges.

2. Step 2: Compare with other data formats

   Key-value pairs, tables, and JSON documents are typical of other NoSQL types, not graph databases.

3. Final Answer:

   A set of nodes connected to 'Alice' by edges labeled 'friend' -> Option B

4. Quick Check:

   Graph query = nodes and edges [OK]

Hint: Graph = nodes connected by edges [OK]

Common Mistakes:

• Expecting JSON documents from graph queries
• Thinking graph data is tabular
• Confusing key-value pairs with graph edges

4. You wrote a query to retrieve data from a column store but got an error. Which mistake likely caused this?

medium

A. Used SQL SELECT syntax without specifying columns

B. Used key-value pairs instead of column names

C. Tried to traverse edges like in a graph database

D. Tried to query nested JSON documents directly

Solution

1. Step 1: Understand column store query requirements

   Column stores require specifying columns in queries; missing columns causes errors.

2. Step 2: Analyze other options

   Nested JSON is for document stores, key-value pairs are for key-value stores, and edge traversal is for graph databases.

3. Final Answer:

   Used SQL SELECT syntax without specifying columns -> Option A

4. Quick Check:

   Column store queries need columns specified [OK]

Hint: Specify columns in column store queries [OK]

Common Mistakes:

• Using document store syntax on column stores
• Trying graph traversal in column stores
• Confusing key-value pairs with columns

5. You need to design a social network app that quickly finds mutual friends between users. Which NoSQL database type is best suited and why?

hard

A. Column database, because it stores data in columns

B. Key-value store, because it is the fastest for any query

C. Document database, because it stores nested user data

D. Graph database, because it efficiently handles relationships

Solution

1. Step 1: Identify the data need for mutual friends

   Finding mutual friends requires exploring relationships between users efficiently.

2. Step 2: Match data need to NoSQL type

   Graph databases specialize in storing and querying relationships like friendships quickly.

3. Step 3: Evaluate other options

   Document stores handle nested data but not relationship queries efficiently; key-value stores are simple but not optimized for relationships; column stores focus on columnar data.

4. Final Answer:

   Graph database, because it efficiently handles relationships -> Option D

5. Quick Check:

   Relationship queries = Graph database [OK]

Hint: Use graph DB for relationship-heavy apps [OK]

Common Mistakes:

• Choosing document DB for relationship queries
• Assuming key-value stores handle complex queries
• Confusing column stores with relationship handling