Bird
Raised Fist0
Microservicessystem_design~10 mins

First microservice architecture diagram in Microservices - Scalability & System Analysis

Choose your learning style10 modes available
LearnWhyDeepArchPracticeChallengeDesignRecallScale

Start learning this pattern below

Jump into concepts and practice - no test required

or
Recommended
Test this pattern10 questions across easy, medium, and hard to know if this pattern is strong
Scalability Analysis - First microservice architecture diagram
Growth Table: Scaling Microservice Architecture
UsersChanges in System
100 usersSingle instance per microservice; simple API gateway; database per service; low latency
10,000 usersMultiple instances per microservice; load balancer added; database replicas for reads; caching introduced
1,000,000 usersMicroservices deployed across multiple regions; database sharding; asynchronous messaging; CDN for static content
100,000,000 usersGlobal multi-cloud deployment; advanced service mesh; automated scaling; event-driven architecture; data partitioning and archiving
First Bottleneck

At small to medium scale, the database becomes the first bottleneck. Each microservice often has its own database, but as user requests grow, the database can struggle to handle the increased query load and data volume. This causes slower responses and potential downtime.

Scaling Solutions
  • Horizontal scaling: Add more instances of microservices behind load balancers to handle more requests.
  • Database replication: Use read replicas to distribute read queries and reduce load on the primary database.
  • Caching: Introduce caches (like Redis) to store frequently accessed data and reduce database hits.
  • Database sharding: Split large databases into smaller parts based on user or data type to improve performance.
  • Asynchronous messaging: Use message queues to decouple services and handle spikes smoothly.
  • CDN: Use Content Delivery Networks to serve static content closer to users, reducing latency and bandwidth.
Back-of-Envelope Cost Analysis
  • At 10,000 users, expect ~1000-5000 requests per second (RPS) depending on user activity.
  • Each microservice instance can handle ~1000-5000 concurrent connections.
  • Database replicas can handle ~5000-10,000 queries per second (QPS) each.
  • Cache can handle ~100,000 operations per second.
  • Network bandwidth: 1 Gbps (~125 MB/s) can support thousands of requests with small payloads.
  • Storage needs grow with user data; plan for scalable storage solutions like cloud object storage.
Interview Tip

When discussing microservice scalability, start by explaining the architecture basics. Then identify the first bottleneck (usually the database). Next, describe step-by-step how to scale each component: services, databases, caching, and network. Use real numbers to show understanding. Finally, mention monitoring and automation for smooth scaling.

Self-Check Question

Your database handles 1000 queries per second (QPS). Traffic grows 10x to 10,000 QPS. What do you do first and why?

Answer: Add read replicas to distribute the increased read load and reduce pressure on the primary database. This is the quickest way to scale database reads without major redesign.

Key Result
The database is the first bottleneck in microservice architectures as user traffic grows; scaling solutions include adding replicas, caching, and sharding to maintain performance.

Practice

(1/5)
1. What is the main role of an API Gateway in a microservice architecture?
easy
A. It stores all the data for the microservices.
B. It routes client requests to the correct microservice.
C. It runs the user interface of the application.
D. It replaces all microservices with a single service.

Solution

  1. Step 1: Understand the API Gateway function

    The API Gateway acts as a single entry point that directs client requests to the appropriate microservice.

  2. Step 2: Compare other options

    Storing data is done by individual services, not the gateway. The UI runs separately, and the gateway does not replace microservices.

  3. Final Answer:

    It routes client requests to the correct microservice. -> Option B

  4. Quick Check:

    API Gateway = Request Router [OK]
Hint: API Gateway directs requests, it does not store data [OK]
Common Mistakes:
  • Thinking API Gateway stores data
  • Confusing API Gateway with UI component
  • Assuming API Gateway replaces microservices
2. Which of the following correctly shows a microservice owning its own data?
easy
A. Multiple microservices share one database directly.
B. Microservices do not use databases at all.
C. Each microservice has its own separate database.
D. All microservices write to a single shared file.

Solution

  1. Step 1: Recall microservice data ownership principle

    Each microservice should own and manage its own database to avoid tight coupling.

  2. Step 2: Evaluate options

    Sharing one database or file breaks independence. Not using databases is unrealistic for data needs.

  3. Final Answer:

    Each microservice has its own separate database. -> Option C

  4. Quick Check:

    Microservice = Own Data Store [OK]
Hint: Microservices keep data separate, no shared DB [OK]
Common Mistakes:
  • Assuming all services share one database
  • Thinking microservices don't need databases
  • Using shared files for data storage
3. Given this simple microservice setup:
Client -> API Gateway -> Service A -> Service B
What happens if Service B is down when Client sends a request?
medium
A. The API Gateway automatically retries Service B until it responds.
B. The API Gateway routes the request to Service B's backup automatically.
C. The client request is handled fully by Service A without contacting Service B.
D. Service A will fail to complete the request and return an error to the client.

Solution

  1. Step 1: Trace request flow with Service B down

    The client request goes through API Gateway to Service A, which calls Service B. If Service B is down, Service A cannot complete its task.

  2. Step 2: Understand failure impact

    Without Service B responding, Service A returns an error back through the API Gateway to the client.

  3. Final Answer:

    Service A will fail to complete the request and return an error to the client. -> Option D

  4. Quick Check:

    Down service causes error response [OK]
Hint: Down service causes error, no automatic retry [OK]
Common Mistakes:
  • Assuming automatic retries by API Gateway
  • Thinking Service A can handle request alone
  • Believing API Gateway has backup routing
4. In this microservice diagram, the API Gateway calls Service A and Service B directly. But Service A calls Service B internally and Service B calls Service A internally.
What is the main problem with this design?
medium
A. It creates a circular dependency between services.
B. API Gateway should not call any services directly.
C. Services should share one database instead.
D. Service A should call Service B, not the other way.

Solution

  1. Step 1: Identify service call relationships

    API Gateway calls both Service A and Service B, and Service A calls Service B and Service B calls Service A, forming a loop.

  2. Step 2: Understand circular dependency issues

    Circular dependencies cause tight coupling and can lead to failures or deadlocks.

  3. Final Answer:

    It creates a circular dependency between services. -> Option A

  4. Quick Check:

    Circular calls = Bad design [OK]
Hint: Avoid circular calls between microservices [OK]
Common Mistakes:
  • Thinking API Gateway shouldn't call services
  • Believing shared database fixes call loops
  • Assuming call direction must be reversed
5. You want to design a microservice architecture for an online store with these needs:
- User service manages user profiles
- Product service manages product info
- Order service creates orders and needs user and product data

Which architecture diagram best fits these requirements?
hard
A. Client -> API Gateway -> User Service, Product Service, Order Service; Order Service calls User and Product Services internally.
B. Client -> API Gateway -> Order Service only; Order Service manages users and products internally.
C. Client -> API Gateway -> User Service and Product Service only; Order Service is part of API Gateway.
D. Client -> API Gateway -> User Service; Product Service calls Order Service; Order Service calls User Service.

Solution

  1. Step 1: Analyze service responsibilities

    User and Product services manage their own data. Order service needs to create orders using data from both.

  2. Step 2: Check communication flow

    Order service calling User and Product services internally keeps responsibilities clear and allows data ownership.

  3. Step 3: Evaluate options

    Client -> API Gateway -> User Service, Product Service, Order Service; Order Service calls User and Product Services internally. matches this design. Others either merge services incorrectly or create wrong call flows.

  4. Final Answer:

    Client -> API Gateway -> User Service, Product Service, Order Service; Order Service calls User and Product Services internally. -> Option A

  5. Quick Check:

    Order service calls needed services [OK]
Hint: Order service calls user and product services internally [OK]
Common Mistakes:
  • Merging all logic into one service
  • Placing order service inside API Gateway
  • Incorrect service call directions