Sidecar proxy pattern in Microservices - System Design Exercise

Design: Sidecar Proxy Pattern for Microservices

Design the sidecar proxy pattern architecture for microservices communication. Out of scope: detailed implementation of microservices business logic.

Functional Requirements

FR1: Enable secure and reliable communication between microservices

FR2: Provide observability features like logging, metrics, and tracing without changing service code

FR3: Support traffic routing, retries, and circuit breaking at the network level

FR4: Allow independent deployment and scaling of sidecar proxies alongside microservices

Non-Functional Requirements

NFR1: Handle up to 10,000 concurrent service-to-service requests

NFR2: Maintain p99 latency under 100ms for proxy operations

NFR3: Ensure 99.9% availability for service communication

NFR4: Minimal performance overhead added by sidecar proxies

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Sidecar proxy container or process

Service mesh control plane

Service registry and discovery

Telemetry and logging systems

Load balancer and circuit breaker modules

Design Patterns

Service mesh architecture

Proxy pattern

Circuit breaker pattern

Sidecar pattern

Observer pattern for telemetry

Reference Architecture

 +----------------+       +----------------+       +----------------+
 | Microservice A |<----->| Sidecar Proxy A |<----->| Service Mesh   |
 +----------------+       +----------------+       | Control Plane  |
                                                      +----------------+
          |                         |                         |
          v                         v                         v
 +----------------+       +----------------+       +----------------+
 | Microservice B |<----->| Sidecar Proxy B |<----->| Telemetry &    |
 +----------------+       +----------------+       | Logging System |
                                                      +----------------+

Components

Sidecar Proxy

Envoy, Linkerd, or custom proxy

Intercepts and manages all network traffic for its microservice, providing routing, retries, security, and telemetry.

Service Mesh Control Plane

Istio, Linkerd control plane

Manages configuration, policies, and service discovery for sidecar proxies.

Microservice

Any microservice framework

Business logic container that communicates through the sidecar proxy.

Telemetry & Logging System

Prometheus, Jaeger, ELK stack

Collects metrics, traces, and logs from sidecar proxies for observability.

Request Flow

1. 1. Microservice sends a request to another microservice via its local sidecar proxy.

2. 2. Sidecar proxy applies routing rules, retries, and circuit breaking before forwarding the request.

3. 3. Request is sent over the network to the destination service's sidecar proxy.

4. 4. Destination sidecar proxy receives the request and forwards it to its local microservice.

5. 5. Sidecar proxies collect telemetry data (latency, errors) and send it to the telemetry system.

6. 6. Service mesh control plane updates routing and policy configurations dynamically.

Database Schema

Not applicable as this pattern focuses on network communication and proxy components rather than data storage.

Scaling Discussion

Bottlenecks

Sidecar proxy CPU and memory usage increases with high request volume

Control plane becomes a bottleneck managing many proxies

Telemetry system overload with high volume of metrics and traces

Network latency added by proxy hops

Solutions

Optimize proxy resource usage and enable horizontal scaling of microservices with sidecars

Use a distributed and scalable control plane architecture

Implement sampling and aggregation in telemetry to reduce data volume

Use efficient protocols and keep proxies lightweight to minimize latency

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain the purpose of the sidecar proxy pattern in microservices

Describe how sidecars enable observability and traffic control without changing service code

Discuss deployment alongside microservices and interaction with the control plane

Highlight benefits like security, retries, and circuit breaking at the network layer

Address scaling challenges and solutions for proxies and control plane

Practice

1. What is the main purpose of the sidecar proxy pattern in microservices architecture?

easy

A. To handle database transactions directly

B. To replace the main service with a proxy for better performance

C. To store data separately from the main service

D. To add features like communication and security without changing the service code

Solution

1. Step 1: Understand the role of sidecar proxy

   The sidecar proxy runs alongside the main service to add extra features such as communication handling, security, and monitoring.

2. Step 2: Identify what it does not do

   It does not replace the service, store data, or handle database transactions directly.

3. Final Answer:

   To add features like communication and security without changing the service code -> Option D

4. Quick Check:

   Sidecar proxy adds features without changing service code = D [OK]

Hint: Sidecar adds features beside service, not inside it [OK]

Common Mistakes:

• Thinking sidecar replaces the main service
• Confusing sidecar with database or storage
• Assuming sidecar handles business logic

2. Which of the following is the correct way to describe the deployment of a sidecar proxy in a microservices environment?

easy

A. Deployed alongside the main service in the same environment or container

B. Deployed as a separate service on a different server

C. Deployed inside the main service codebase

D. Deployed only on the client side

Solution

1. Step 1: Understand sidecar deployment

   The sidecar proxy runs alongside the main service, usually in the same environment or container, to intercept and manage traffic.

2. Step 2: Eliminate incorrect options

   It is not deployed as a separate service on a different server, nor inside the main service code, nor only on the client side.

3. Final Answer:

   Deployed alongside the main service in the same environment or container -> Option A

4. Quick Check:

   Sidecar runs alongside service = A [OK]

Hint: Sidecar runs side-by-side, not separately or inside code [OK]

Common Mistakes:

• Thinking sidecar is a separate remote service
• Confusing sidecar with code library inside service
• Assuming sidecar runs only on client machines

3. Consider this simplified request flow in a microservice using a sidecar proxy:

Client -> Sidecar Proxy -> Service -> Sidecar Proxy -> Client

What is the main benefit of this flow?

medium

A. The sidecar proxy can handle retries, security checks, and logging without changing the service

B. The service can directly communicate with the client without any proxy

C. The sidecar proxy replaces the service logic for faster processing

D. The client bypasses the sidecar proxy for faster response

Solution

1. Step 1: Analyze the request flow with sidecar proxy

   The sidecar proxy intercepts requests and responses, allowing it to add features like retries, security checks, and logging transparently.

2. Step 2: Understand the benefit of this interception

   This keeps the service code simple and focused on business logic, while the sidecar handles cross-cutting concerns.

3. Final Answer:

   The sidecar proxy can handle retries, security checks, and logging without changing the service -> Option A

4. Quick Check:

   Sidecar manages extra tasks transparently = A [OK]

Hint: Sidecar handles extras, service stays simple [OK]

Common Mistakes:

• Thinking sidecar replaces service logic
• Assuming client talks directly to service
• Believing sidecar slows down response by bypassing

4. A developer notices that the sidecar proxy is not forwarding requests to the main service correctly. Which of the following is the most likely cause?

medium

A. The main service code has a syntax error

B. The client is not sending requests

C. The sidecar proxy configuration is missing the service's local address

D. The database is down

Solution

1. Step 1: Identify sidecar proxy forwarding issue

   If the sidecar proxy does not forward requests, it is often due to incorrect or missing configuration about where the main service is located.

2. Step 2: Rule out unrelated causes

   Syntax errors in service code, client not sending requests, or database issues do not directly cause proxy forwarding failures.

3. Final Answer:

   The sidecar proxy configuration is missing the service's local address -> Option C

4. Quick Check:

   Proxy forwarding fails if service address missing = B [OK]

Hint: Check proxy config for service address first [OK]

Common Mistakes:

• Blaming service code syntax errors
• Assuming client or database issues cause proxy failure
• Ignoring proxy configuration details

5. You want to add monitoring and security features to multiple microservices without changing their code. How does the sidecar proxy pattern help solve this problem at scale?

hard

A. By centralizing monitoring and security in a single proxy for all services

B. By deploying a sidecar proxy with each service instance to handle monitoring and security independently

C. By rewriting each service to include monitoring and security code

D. By removing proxies and letting clients handle monitoring

Solution

1. Step 1: Understand scaling with sidecar proxies

   Deploying a sidecar proxy alongside each service instance allows independent handling of monitoring and security without modifying service code.

2. Step 2: Compare with other options

   Rewriting services is costly and error-prone; centralizing in one proxy creates a bottleneck; removing proxies loses control.

3. Final Answer:

   By deploying a sidecar proxy with each service instance to handle monitoring and security independently -> Option B

4. Quick Check:

   Sidecar per service instance scales features independently = C [OK]

Hint: Sidecar per service instance scales features well [OK]

Common Mistakes:

• Thinking one proxy can handle all services centrally
• Assuming code changes are needed for features
• Ignoring scalability and bottleneck issues