Anti-patterns (distributed monolith, chatty services) in Microservices - Scalability & System Analysis

Scalability Analysis - Anti-patterns (distributed monolith, chatty services)

Growth Table: Impact of Anti-patterns at Different Scales

Users	System Behavior	Performance Impact	Operational Complexity
100 users	Services communicate frequently but delays are small	Minor latency, manageable	Low complexity, easy debugging
10,000 users	Increased inter-service calls cause noticeable delays	Latency grows, throughput drops	Harder to trace issues, deployment slower
1,000,000 users	High network chatter causes bottlenecks, cascading failures	Severe latency, timeouts, resource exhaustion	Complex debugging, frequent outages
100,000,000 users	System behaves like a monolith, scaling fails	System crashes, unresponsive services	Very high operational cost, near impossible to maintain

First Bottleneck

The first bottleneck is the network and service communication layer. Because services call each other too often (chatty services), the network becomes saturated and latency spikes. This leads to slow responses and cascading failures. Also, the distributed monolith anti-pattern means services are tightly coupled, so a failure in one service blocks others, reducing overall system availability.

Scaling Solutions

• Reduce Chatty Calls: Combine related operations into fewer calls to reduce network overhead.
• API Gateway & Aggregation: Use an API gateway to aggregate multiple service calls into one.
• Event-Driven Architecture: Use asynchronous messaging to decouple services and reduce direct calls.
• Service Boundaries: Redesign services to be more independent, avoiding distributed monolith.
• Caching: Cache frequent data to avoid repeated calls.
• Load Balancing & Horizontal Scaling: Add more instances to handle increased load.
• Monitoring & Tracing: Implement distributed tracing to identify and fix chatty patterns.

Back-of-Envelope Cost Analysis

Assuming 1 million users generate 10 requests per second each, total requests = 10 million QPS.

• Each chatty service call multiplies network traffic by 5x → 50 million calls per second.
• Network bandwidth needed: If each call is 1 KB, total bandwidth = 50 GB/s (~400 Gbps), which is very high.
• Database load increases with each call, risking overload beyond 10,000 QPS per instance.
• CPU and memory usage spike due to handling many small calls.

Costs rise sharply due to inefficient communication and resource usage.

Interview Tip

When discussing scalability, first identify if the system has tight coupling or chatty communication. Explain how these cause bottlenecks. Then propose concrete solutions like reducing calls, using async messaging, and redesigning service boundaries. Show understanding of trade-offs and operational impacts.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Introduce read replicas and caching to reduce load on the primary database before scaling vertically or horizontally.

Key Result

Chatty services and distributed monoliths cause network and coupling bottlenecks that limit scalability; reducing inter-service calls and decoupling services are key to scaling efficiently.

Practice

1. Which of the following best describes a distributed monolith in microservices architecture?

easy

A. Services are fully independent and communicate rarely.

B. Services are tightly coupled and require coordinated deployment.

C. Services use asynchronous messaging to reduce latency.

D. Services are stateless and scale automatically.

Solution

1. Step 1: Understand distributed monolith characteristics

   A distributed monolith looks like microservices but behaves like a single app with tight coupling.

2. Step 2: Identify deployment and coupling issues

   Such services require coordinated deployment and cannot scale independently.

3. Final Answer:

   Services are tightly coupled and require coordinated deployment. -> Option B

4. Quick Check:

   Distributed monolith = tight coupling [OK]

Hint: Distributed monolith means tight coupling, not independence [OK]

Common Mistakes:

• Confusing distributed monolith with loosely coupled microservices
• Thinking distributed monolith scales independently
• Assuming distributed monolith uses asynchronous calls

2. Which syntax correctly describes a common symptom of chatty services in microservices communication?

easy

A. Service A uses event-driven messaging to notify Service B.

B. Service A calls Service B once per user request.

C. Service A calls Service B multiple times per user request.

D. Service A caches data to reduce calls to Service B.

Solution

1. Step 1: Define chatty services behavior

   Chatty services make many small calls between services per user request.

2. Step 2: Identify the correct syntax describing chatty calls

   Multiple calls per request indicate chatty communication.

3. Final Answer:

   Service A calls Service B multiple times per user request. -> Option C

4. Quick Check:

   Chatty services = many calls [OK]

Hint: Chatty means many calls, not just one [OK]

Common Mistakes:

• Choosing event-driven messaging as chatty behavior
• Assuming caching causes chatty services
• Thinking one call per request is chatty

3. Given a microservices system where Service A calls Service B 5 times and Service B calls Service C 3 times per user request, what is the total number of service calls triggered by one user request?

medium

A. 20

B. 15

C. 30

D. 8

Solution

1. Step 1: Calculate calls from Service A to B

   Service A calls Service B 5 times per request.

2. Step 2: Calculate calls from Service B to C triggered by A's calls

   Each of the 5 calls from A causes 3 calls from B to C, so 5 * 3 = 15 calls.

3. Step 3: Sum all calls

   Total calls = 5 (A->B) + 15 (B->C) = 20 calls.

4. Final Answer:

   20 -> Option A

5. Quick Check:

   5 + (5*3) = 20 [OK]

Hint: Multiply nested calls, then add all [OK]

Common Mistakes:

• Adding 5 + 3 instead of multiplying
• Ignoring nested calls from B to C
• Choosing sum as 18 instead of 20

4. You notice your microservices system has high latency due to many small synchronous calls between services. Which change would best fix this chatty service anti-pattern?

medium

A. Use asynchronous messaging or batch requests to reduce calls.

B. Combine tightly coupled services into a single service.

C. Add more synchronous calls to improve data freshness.

D. Increase the number of service instances to handle load.

Solution

1. Step 1: Identify chatty service problem

   Many small synchronous calls cause latency and network overhead.

2. Step 2: Choose solution to reduce call frequency

   Using asynchronous messaging or batching reduces calls and latency.

3. Final Answer:

   Use asynchronous messaging or batch requests to reduce calls. -> Option A

4. Quick Check:

   Reduce calls with async or batching [OK]

Hint: Reduce calls by batching or async messaging [OK]

Common Mistakes:

• Combining services creates distributed monolith
• Adding more sync calls worsens latency
• Scaling instances doesn't reduce call count

5. A company has a microservices system suffering from both distributed monolith and chatty services anti-patterns. Which combined approach best improves scalability and deployment independence?

hard

A. Merge all services into one large application to simplify deployment.

B. Increase hardware resources and add load balancers to handle traffic.

C. Use synchronous REST calls extensively to keep services tightly connected.

D. Refactor services to reduce dependencies and use asynchronous communication.

Solution

1. Step 1: Address distributed monolith by reducing dependencies

   Refactoring services to be loosely coupled allows independent deployment and scaling.

2. Step 2: Fix chatty services by adopting asynchronous communication

   Using async messaging reduces frequent synchronous calls and network overhead.

3. Step 3: Combine both improvements for better scalability and independence

   This combined approach solves both anti-patterns effectively.

4. Final Answer:

   Refactor services to reduce dependencies and use asynchronous communication. -> Option D

5. Quick Check:

   Loose coupling + async = scalable microservices [OK]

Hint: Loose coupling + async communication fixes both issues [OK]

Common Mistakes:

• Merging services worsens distributed monolith
• Adding hardware doesn't fix design flaws
• Using more sync calls increases chatty problems