Database decomposition strategy in Microservices - Scalability & System Analysis

Scalability Analysis - Database decomposition strategy

Growth Table: Database Decomposition Strategy

Users	Data Volume	Database Setup	Challenges	Changes Needed
100 users	Small (MBs)	Single monolithic database	Simple queries, low latency	None, simple design works
10,000 users	Medium (GBs)	Monolithic DB with read replicas	Read scaling, some write contention	Add read replicas, caching
1,000,000 users	Large (TBs)	Decompose DB by service (microservices)	Write bottlenecks, complex joins, latency	Split DB by domain, use separate DB per microservice
100,000,000 users	Very Large (PBs)	Sharded and decomposed DBs per service	Cross-service data consistency, network latency	Shard databases, async communication, event sourcing

First Bottleneck

At around 1 million users, the monolithic database becomes the first bottleneck. It struggles with write throughput and complex joins across large tables. This causes slow response times and limits scaling.

Scaling Solutions

• Database decomposition: Split the monolithic database into multiple smaller databases aligned with microservices. Each service owns its data.
• Horizontal scaling: Add more database instances for each microservice to distribute load.
• Caching: Use caches to reduce database reads for frequently accessed data.
• Sharding: Partition large databases by key ranges or user IDs to spread data across servers.
• Async communication: Use event-driven patterns to sync data between services without blocking.

Back-of-Envelope Cost Analysis

• At 1M users, expect ~10,000 QPS (queries per second) on the database.
• A single PostgreSQL instance handles ~5,000 QPS, so two or more DBs needed.
• Storage grows to terabytes; decomposed DBs reduce single DB size.
• Network bandwidth increases due to inter-service communication; plan for 1 Gbps+ links.
• Caching reduces DB load by 30-50%, saving costs and improving latency.

Interview Tip

Start by explaining the limits of a monolithic database as users grow. Then describe how splitting data by service boundaries helps scale. Discuss trade-offs like data consistency and complexity. Finally, mention caching, sharding, and async communication as further steps.

Self Check

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add read replicas and implement caching to reduce load on the primary database before considering decomposition or sharding.

Key Result

Database decomposition becomes essential at around 1 million users to overcome monolithic DB bottlenecks by splitting data per microservice, enabling horizontal scaling and reducing latency.

Practice

1. Which of the following best describes vertical decomposition in database design for microservices?

easy

A. Dividing a database by rows to distribute data across multiple databases

B. Combining multiple databases into one large database

C. Separating databases based on geographic location

D. Splitting a database by grouping related tables or columns into separate databases

Solution

1. Step 1: Understand vertical decomposition

   Vertical decomposition means splitting a database by grouping related tables or columns, often by business capability or domain.

2. Step 2: Compare with other options

   Horizontal decomposition splits by rows, geographic is location-based, and combining is the opposite of decomposition.

3. Final Answer:

   Splitting a database by grouping related tables or columns into separate databases -> Option D

4. Quick Check:

   Vertical decomposition = splitting by columns/tables [OK]

Hint: Vertical = split by columns or tables, horizontal = split by rows [OK]

Common Mistakes:

• Confusing vertical with horizontal decomposition
• Thinking vertical means geographic split
• Assuming decomposition means combining databases

2. Which of the following is the correct description of horizontal decomposition in microservices database design?

easy

A. Dividing data by rows, such as by customer or region

B. Splitting data by columns or tables based on functionality

C. Merging multiple databases into one for simplicity

D. Separating databases by different database engines

Solution

1. Step 1: Define horizontal decomposition

   Horizontal decomposition splits data by rows, for example, dividing customers by region or user ID ranges.

2. Step 2: Eliminate incorrect options

   Splitting data by columns or tables based on functionality describes vertical decomposition, C is merging (not decomposition), and D is about engines, not decomposition strategy.

3. Final Answer:

   Dividing data by rows, such as by customer or region -> Option A

4. Quick Check:

   Horizontal decomposition = split by rows [OK]

Hint: Horizontal = split by rows, vertical = split by columns [OK]

Common Mistakes:

• Mixing horizontal with vertical decomposition
• Thinking horizontal means merging databases
• Confusing database engine separation with decomposition

3. Consider a microservices system where the user database is split by region using horizontal decomposition. If a query requests all users from Europe, which database(s) will be queried?

medium

A. Only the database shard containing European users

B. All database shards regardless of region

C. Only the database shard containing North American users

D. A combined database with all users merged

Solution

1. Step 1: Understand horizontal decomposition by region

   Horizontal decomposition splits data by rows, so each shard holds users from a specific region.

2. Step 2: Identify which shard to query

   Querying European users targets only the shard holding European data, not others.

3. Final Answer:

   Only the database shard containing European users -> Option A

4. Quick Check:

   Query targets relevant shard only [OK]

Hint: Query only the shard holding requested data region [OK]

Common Mistakes:

• Querying all shards unnecessarily
• Querying wrong region shard
• Assuming data is merged in one database

4. A microservices team decomposed their database vertically but notices frequent cross-service joins causing latency. What is the likely cause and fix?

medium

A. Cause: Using NoSQL instead of SQL; Fix: Switch to SQL databases

B. Cause: Horizontal decomposition; Fix: Merge databases into one

C. Cause: Poor vertical decomposition causing cross-service joins; Fix: Redesign to reduce cross-service dependencies

D. Cause: Too many database shards; Fix: Increase shards further

Solution

1. Step 1: Identify problem with vertical decomposition

   Vertical decomposition splits by tables/domains, but if services need to join data often, it causes latency.

2. Step 2: Recommend fix

   Redesign to reduce cross-service joins by better domain boundaries or data duplication to avoid latency.

3. Final Answer:

   Poor vertical decomposition causing cross-service joins; Fix: Redesign to reduce cross-service dependencies -> Option C

4. Quick Check:

   Cross-service joins cause latency; fix by better decomposition [OK]

Hint: Cross-service joins mean bad vertical split; redesign domains [OK]

Common Mistakes:

• Confusing horizontal with vertical decomposition issues
• Thinking merging databases fixes latency
• Blaming database type instead of design

5. A company wants to scale their microservices database by splitting user data by country (horizontal) and splitting user profile and orders into separate databases (vertical). What is the best approach to handle queries that need both profile and order data for users in a specific country?

hard

A. Perform cross-database joins directly on all shards for each country

B. Use API composition to aggregate data from profile and order services after querying country-specific shards

C. Merge profile and order data into a single database shard per country

D. Store all user data in one large database to avoid complexity

Solution

1. Step 1: Understand combined vertical and horizontal decomposition

   Data is split horizontally by country and vertically by data type (profile, orders), so data is in different shards and databases.

2. Step 2: Choose best query approach

   Cross-database joins are expensive and complex; merging data loses benefits. API composition aggregates data from services after querying relevant shards efficiently.

3. Final Answer:

   Use API composition to aggregate data from profile and order services after querying country-specific shards -> Option B

4. Quick Check:

   API composition handles multi-db queries efficiently [OK]

Hint: Use API composition to combine data from vertical and horizontal splits [OK]

Common Mistakes:

• Trying cross-database joins causing latency
• Merging databases losing scalability
• Ignoring decomposition benefits for simplicity