Anti-patterns (distributed monolith, chatty services) in Microservices - System Design Exercise

Design: Microservices Architecture with Anti-patterns

Focus on service design and communication patterns. Database design and UI are out of scope.

Functional Requirements

FR1: Design a microservices-based system for an e-commerce platform

FR2: Services should handle user management, product catalog, order processing, and payment

FR3: Each service should be independently deployable and scalable

FR4: Services must communicate to fulfill user requests

Non-Functional Requirements

NFR1: System should support 10,000 concurrent users

NFR2: API response time p99 under 300ms

NFR3: Availability target 99.9% uptime

NFR4: Avoid tight coupling between services

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

API Gateway

Service Registry and Discovery

Message Broker for asynchronous communication

Individual microservices with own databases

Load Balancer

Design Patterns

Service Mesh

Event-driven architecture

Circuit Breaker

Bulkhead pattern

API Gateway pattern

Reference Architecture

          +----------------+          +----------------+
          |  API Gateway   |          |  Service Mesh  |
          +-------+--------+          +-------+--------+
                  |                           |
    +-------------+-------------+-------------+-------------+
    |                           |                           |
+---v---+                   +---v---+                   +---v---+
| User  |                   | Order |                   | Product|
|Service|                   |Service|                   |Service |
+---+---+                   +---+---+                   +---+---+
    |                           |                           |
+---v---+                   +---v---+                   +---v---+
| User  |                   | Order |                   | Product|
| DB    |                   | DB    |                   | DB     |
+-------+                   +-------+                   +-------+

Note: Services communicate asynchronously via message broker to reduce chatty calls.

Components

API Gateway

Nginx or Kong

Single entry point for clients, routes requests to services

User Service

Spring Boot microservice

Handles user registration, authentication, profile management

Order Service

Node.js microservice

Manages order creation, updates, and status

Product Service

Go microservice

Manages product catalog and inventory

Message Broker

Apache Kafka or RabbitMQ

Enables asynchronous communication between services to reduce chatty calls

Service Mesh

Istio or Linkerd

Manages service-to-service communication, retries, and circuit breaking

Databases

PostgreSQL per service

Each service owns its own database to avoid tight coupling

Request Flow

1. Client sends request to API Gateway

2. API Gateway routes request to appropriate microservice

3. Microservice processes request using its own database

4. If data from another service is needed, microservice publishes event to message broker

5. Other services consume events asynchronously to update their state or trigger actions

6. Service Mesh manages communication reliability and security

7. API Gateway returns response to client

Database Schema

UserService: User(id PK, name, email, password_hash) OrderService: Order(id PK, user_id FK, product_id FK, quantity, status) ProductService: Product(id PK, name, description, price, stock_quantity) Each service owns its schema with no shared tables to avoid distributed monolith.

Scaling Discussion

Bottlenecks

Excessive synchronous calls between services causing high latency (chatty services)

Tight coupling due to shared databases leading to deployment and scaling issues (distributed monolith)

Single API Gateway becoming a bottleneck under high load

Message broker overload if event volume grows rapidly

Solutions

Adopt asynchronous communication with event-driven architecture to reduce chatty calls

Ensure each service owns its database and data model to avoid distributed monolith

Scale API Gateway horizontally with load balancers

Partition message broker topics and scale brokers horizontally

Use service mesh features like circuit breakers and retries to improve resilience

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing architecture and explaining anti-patterns, 10 minutes discussing scaling and trade-offs, 5 minutes for questions

Explain the difference between a distributed monolith and a true microservices architecture

Highlight problems caused by chatty services and how asynchronous communication helps

Discuss importance of service autonomy and database ownership

Mention use of service mesh for communication reliability

Address scaling bottlenecks realistically with practical solutions

Practice

1. Which of the following best describes a distributed monolith in microservices architecture?

easy

A. Services are fully independent and communicate rarely.

B. Services are tightly coupled and require coordinated deployment.

C. Services use asynchronous messaging to reduce latency.

D. Services are stateless and scale automatically.

Solution

1. Step 1: Understand distributed monolith characteristics

   A distributed monolith looks like microservices but behaves like a single app with tight coupling.

2. Step 2: Identify deployment and coupling issues

   Such services require coordinated deployment and cannot scale independently.

3. Final Answer:

   Services are tightly coupled and require coordinated deployment. -> Option B

4. Quick Check:

   Distributed monolith = tight coupling [OK]

Hint: Distributed monolith means tight coupling, not independence [OK]

Common Mistakes:

• Confusing distributed monolith with loosely coupled microservices
• Thinking distributed monolith scales independently
• Assuming distributed monolith uses asynchronous calls

2. Which syntax correctly describes a common symptom of chatty services in microservices communication?

easy

A. Service A uses event-driven messaging to notify Service B.

B. Service A calls Service B once per user request.

C. Service A calls Service B multiple times per user request.

D. Service A caches data to reduce calls to Service B.

Solution

1. Step 1: Define chatty services behavior

   Chatty services make many small calls between services per user request.

2. Step 2: Identify the correct syntax describing chatty calls

   Multiple calls per request indicate chatty communication.

3. Final Answer:

   Service A calls Service B multiple times per user request. -> Option C

4. Quick Check:

   Chatty services = many calls [OK]

Hint: Chatty means many calls, not just one [OK]

Common Mistakes:

• Choosing event-driven messaging as chatty behavior
• Assuming caching causes chatty services
• Thinking one call per request is chatty

3. Given a microservices system where Service A calls Service B 5 times and Service B calls Service C 3 times per user request, what is the total number of service calls triggered by one user request?

medium

A. 20

B. 15

C. 30

D. 8

Solution

1. Step 1: Calculate calls from Service A to B

   Service A calls Service B 5 times per request.

2. Step 2: Calculate calls from Service B to C triggered by A's calls

   Each of the 5 calls from A causes 3 calls from B to C, so 5 * 3 = 15 calls.

3. Step 3: Sum all calls

   Total calls = 5 (A->B) + 15 (B->C) = 20 calls.

4. Final Answer:

   20 -> Option A

5. Quick Check:

   5 + (5*3) = 20 [OK]

Hint: Multiply nested calls, then add all [OK]

Common Mistakes:

• Adding 5 + 3 instead of multiplying
• Ignoring nested calls from B to C
• Choosing sum as 18 instead of 20

4. You notice your microservices system has high latency due to many small synchronous calls between services. Which change would best fix this chatty service anti-pattern?

medium

A. Use asynchronous messaging or batch requests to reduce calls.

B. Combine tightly coupled services into a single service.

C. Add more synchronous calls to improve data freshness.

D. Increase the number of service instances to handle load.

Solution

1. Step 1: Identify chatty service problem

   Many small synchronous calls cause latency and network overhead.

2. Step 2: Choose solution to reduce call frequency

   Using asynchronous messaging or batching reduces calls and latency.

3. Final Answer:

   Use asynchronous messaging or batch requests to reduce calls. -> Option A

4. Quick Check:

   Reduce calls with async or batching [OK]

Hint: Reduce calls by batching or async messaging [OK]

Common Mistakes:

• Combining services creates distributed monolith
• Adding more sync calls worsens latency
• Scaling instances doesn't reduce call count

5. A company has a microservices system suffering from both distributed monolith and chatty services anti-patterns. Which combined approach best improves scalability and deployment independence?

hard

A. Merge all services into one large application to simplify deployment.

B. Increase hardware resources and add load balancers to handle traffic.

C. Use synchronous REST calls extensively to keep services tightly connected.

D. Refactor services to reduce dependencies and use asynchronous communication.

Solution

1. Step 1: Address distributed monolith by reducing dependencies

   Refactoring services to be loosely coupled allows independent deployment and scaling.

2. Step 2: Fix chatty services by adopting asynchronous communication

   Using async messaging reduces frequent synchronous calls and network overhead.

3. Step 3: Combine both improvements for better scalability and independence

   This combined approach solves both anti-patterns effectively.

4. Final Answer:

   Refactor services to reduce dependencies and use asynchronous communication. -> Option D

5. Quick Check:

   Loose coupling + async = scalable microservices [OK]

Hint: Loose coupling + async communication fixes both issues [OK]

Common Mistakes:

• Merging services worsens distributed monolith
• Adding hardware doesn't fix design flaws
• Using more sync calls increases chatty problems