Why service mesh manages inter-service traffic in Microservices - Why This Architecture

Problem Statement

When multiple microservices communicate directly, managing their interactions becomes complex and error-prone. Without a centralized control, issues like inconsistent security policies, unreliable communication, and difficulty in monitoring arise, leading to failures and degraded user experience.

Solution

A service mesh introduces a dedicated infrastructure layer that transparently manages all inter-service traffic. It handles routing, load balancing, security, and observability by deploying lightweight proxies alongside each service, ensuring consistent and reliable communication without changing service code.

Architecture

Service A

→Sidecar A

↓

Service B

This diagram shows two microservices, each paired with a sidecar proxy. All inter-service calls pass through these proxies, which manage traffic, security, and monitoring.

Trade-offs

✓ Pros

→

Centralizes control of inter-service communication policies without modifying service code.

→

Improves security by enforcing consistent authentication and encryption between services.

→

Enhances observability with detailed metrics and tracing of service interactions.

→

Enables advanced traffic management like retries, circuit breaking, and canary deployments.

✗ Cons

→

Adds operational complexity by introducing additional infrastructure components to manage.

→

Increases resource consumption due to sidecar proxies running alongside each service.

→

Can introduce latency overhead because of proxy processing on every request.

When running a microservices architecture with dozens or more services requiring secure, observable, and reliable communication at scale.

When the system has only a few services with simple communication needs, or when resource constraints make sidecar overhead unacceptable.

Real World Examples

Google

Uses Istio service mesh to manage traffic between thousands of microservices in their cloud platform, ensuring security and observability.

Lyft

Developed Envoy proxy and uses service mesh to handle complex routing and resilience in their ride-sharing microservices.

IBM

Implements service mesh to provide secure and reliable communication in hybrid cloud microservices deployments.

Alternatives

API Gateway

Manages traffic at the edge for external clients, not between internal microservices.

Use when: When controlling and securing external client access is the primary concern.

Client-side Load Balancing

Each service handles routing logic internally without a dedicated proxy layer.

Use when: When the system is small and adding a service mesh is too complex.

Summary

Service mesh solves the complexity of managing communication between many microservices by adding a transparent proxy layer.

It improves security, observability, and traffic control without changing service code.

However, it adds infrastructure overhead and is best suited for large-scale microservices environments.

Practice

1. Why does a service mesh manage inter-service traffic in a microservices architecture?

easy

A. To improve security, reliability, and observability between services

B. To replace the need for a database in microservices

C. To write the business logic inside each service

D. To increase the size of each service for better performance

Solution

1. Step 1: Understand the role of service mesh

   A service mesh controls how services communicate, focusing on security, reliability, and monitoring.

2. Step 2: Identify what service mesh does not do

   It does not replace databases or add business logic; it manages traffic between services.

3. Final Answer:

   To improve security, reliability, and observability between services -> Option A

4. Quick Check:

   Service mesh manages traffic for security and reliability = A [OK]

Hint: Service mesh controls communication, not business logic or storage [OK]

Common Mistakes:

• Thinking service mesh replaces databases
• Confusing service mesh with application code
• Assuming service mesh increases service size

2. Which syntax correctly describes how a service mesh uses sidecar proxies?

easy

A. database -> service -> sidecar proxy

B. service -> sidecar proxy -> other service

C. sidecar proxy -> service -> database

D. service <- database <- sidecar proxy

Solution

1. Step 1: Understand sidecar proxy role

   Sidecar proxies sit alongside services to intercept and manage traffic between services.

2. Step 2: Identify correct traffic flow

   Traffic flows from the service through its sidecar proxy to the other service.

3. Final Answer:

   service -> sidecar proxy -> other service -> Option B

4. Quick Check:

   Sidecar proxies manage traffic between services = D [OK]

Hint: Sidecar proxies sit next to services, managing outgoing traffic [OK]

Common Mistakes:

• Confusing database direction with sidecar proxy
• Reversing traffic flow arrows
• Mixing service and database roles

3. Given this simplified service mesh setup, what is the expected behavior when Service A calls Service B and Service B is temporarily down?

Service A -> Sidecar Proxy A -> Sidecar Proxy B -> Service B

Options:

medium

A. The call fails immediately with no retries

B. Service A handles retries without sidecar involvement

C. Sidecar Proxy A retries the call automatically before failing

D. Sidecar Proxy B forwards the call to a database instead

Solution

1. Step 1: Recognize retry feature in service mesh

   Service mesh sidecar proxies can automatically retry failed calls to improve reliability.

2. Step 2: Identify which proxy handles retries

   Sidecar Proxy A, managing outgoing traffic from Service A, retries the call before reporting failure.

3. Final Answer:

   Sidecar Proxy A retries the call automatically before failing -> Option C

4. Quick Check:

   Sidecar proxies handle retries to improve reliability = B [OK]

Hint: Sidecar proxies retry failed calls automatically [OK]

Common Mistakes:

• Assuming no retries happen
• Thinking service code retries instead
• Confusing proxy roles with database

4. You configured a service mesh but notice that traffic between services is not encrypted. What is the most likely cause?

medium

A. Service mesh does not support encryption

B. Services are using HTTP instead of HTTPS internally

C. The database connection is not encrypted

D. Sidecar proxies are not enabled to handle TLS encryption

Solution

1. Step 1: Understand encryption in service mesh

   Service mesh uses sidecar proxies to encrypt traffic between services using TLS.

2. Step 2: Identify common misconfiguration

   If sidecar proxies are not configured or enabled for TLS, traffic remains unencrypted.

3. Final Answer:

   Sidecar proxies are not enabled to handle TLS encryption -> Option D

4. Quick Check:

   Encryption depends on sidecar proxy TLS setup = A [OK]

Hint: Check sidecar proxy TLS settings for encryption issues [OK]

Common Mistakes:

• Blaming service internal HTTP usage
• Confusing database encryption with service traffic
• Assuming service mesh lacks encryption feature

5. In a microservices system using a service mesh, how does the mesh help when one service experiences intermittent failures?

hard

A. It automatically retries requests, routes around failures, and collects metrics for monitoring

B. It stops all traffic to the failing service until manually restarted

C. It merges the failing service into other services to avoid downtime

D. It disables sidecar proxies to reduce overhead during failures

Solution

1. Step 1: Identify service mesh features for failure handling

   Service mesh retries requests, performs circuit breaking (routing around failures), and gathers metrics.

2. Step 2: Understand what service mesh does not do

   It does not stop all traffic, merge services, or disable proxies during failures.

3. Final Answer:

   It automatically retries requests, routes around failures, and collects metrics for monitoring -> Option A

4. Quick Check:

   Service mesh improves reliability with retries and monitoring = C [OK]

Hint: Service mesh retries and monitors to handle failures smoothly [OK]

Common Mistakes:

• Thinking mesh stops traffic completely
• Believing mesh merges services automatically
• Assuming proxies are disabled on failure