Sidecar proxy pattern in Microservices - System Design Exercise

Design: Sidecar Proxy Pattern for Microservices

Design the sidecar proxy pattern architecture for microservices communication. Out of scope: detailed implementation of microservices business logic.

Functional Requirements

FR1: Enable secure and reliable communication between microservices

FR2: Provide observability features like logging, metrics, and tracing without changing service code

FR3: Support traffic routing, retries, and circuit breaking at the network level

FR4: Allow independent deployment and scaling of sidecar proxies alongside microservices

Non-Functional Requirements

NFR1: Handle up to 10,000 concurrent service-to-service requests

NFR2: Maintain p99 latency under 100ms for proxy operations

NFR3: Ensure 99.9% availability for service communication

NFR4: Minimal performance overhead added by sidecar proxies

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Sidecar proxy container or process

Service mesh control plane

Service registry and discovery

Telemetry and logging systems

Load balancer and circuit breaker modules

Design Patterns

Service mesh architecture

Proxy pattern

Circuit breaker pattern

Sidecar pattern

Observer pattern for telemetry

Reference Architecture

 +----------------+       +----------------+       +----------------+
 | Microservice A |<----->| Sidecar Proxy A |<----->| Service Mesh   |
 +----------------+       +----------------+       | Control Plane  |
                                                      +----------------+
          |                         |                         |
          v                         v                         v
 +----------------+       +----------------+       +----------------+
 | Microservice B |<----->| Sidecar Proxy B |<----->| Telemetry &    |
 +----------------+       +----------------+       | Logging System |
                                                      +----------------+

Components

Sidecar Proxy

Envoy, Linkerd, or custom proxy

Intercepts and manages all network traffic for its microservice, providing routing, retries, security, and telemetry.

Service Mesh Control Plane

Istio, Linkerd control plane

Manages configuration, policies, and service discovery for sidecar proxies.

Microservice

Any microservice framework

Business logic container that communicates through the sidecar proxy.

Telemetry & Logging System

Prometheus, Jaeger, ELK stack

Collects metrics, traces, and logs from sidecar proxies for observability.

Request Flow

1. 1. Microservice sends a request to another microservice via its local sidecar proxy.

2. 2. Sidecar proxy applies routing rules, retries, and circuit breaking before forwarding the request.

3. 3. Request is sent over the network to the destination service's sidecar proxy.

4. 4. Destination sidecar proxy receives the request and forwards it to its local microservice.

5. 5. Sidecar proxies collect telemetry data (latency, errors) and send it to the telemetry system.

6. 6. Service mesh control plane updates routing and policy configurations dynamically.

Database Schema

Not applicable as this pattern focuses on network communication and proxy components rather than data storage.

Scaling Discussion

Bottlenecks

Sidecar proxy CPU and memory usage increases with high request volume

Control plane becomes a bottleneck managing many proxies

Telemetry system overload with high volume of metrics and traces

Network latency added by proxy hops

Solutions

Optimize proxy resource usage and enable horizontal scaling of microservices with sidecars

Use a distributed and scalable control plane architecture

Implement sampling and aggregation in telemetry to reduce data volume

Use efficient protocols and keep proxies lightweight to minimize latency

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain the purpose of the sidecar proxy pattern in microservices

Describe how sidecars enable observability and traffic control without changing service code

Discuss deployment alongside microservices and interaction with the control plane

Highlight benefits like security, retries, and circuit breaking at the network layer

Address scaling challenges and solutions for proxies and control plane