Sidecar proxy pattern in Microservices - System Design Guide

Problem Statement

When microservices communicate directly, each service must implement complex networking features like retries, load balancing, and security. This leads to duplicated code, inconsistent behavior, and harder maintenance. Also, if a service crashes or is updated, network communication can break or degrade without a clear isolation.

Solution

The sidecar proxy pattern solves this by deploying a helper proxy alongside each microservice instance. This proxy handles all network communication tasks such as routing, retries, and security, transparently to the service. The service only talks to its local sidecar, which manages external communication, isolating network logic from business logic.

Architecture

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│   Service A   │──────▶│ Sidecar Proxy │──────▶│   Service B   │
│ (business     │       │ (networking   │       │ (business     │
│  logic only)  │       │  logic)       │       │  logic only)  │
└───────────────┘       └───────────────┘       └───────────────┘
       │                      │                        │
       │                      │                        │
       │ Local communication   │ External communication │
       ▼                      ▼                        ▼

This diagram shows each microservice paired with a sidecar proxy. The service communicates locally with its sidecar, which manages all external network calls to other services.

Trade-offs

✓ Pros

→

Centralizes networking features like retries, load balancing, and security in the sidecar, reducing duplicated code.

→

Improves service isolation by separating business logic from network concerns.

→

Enables consistent communication policies across all services without changing service code.

→

Simplifies updates to networking logic by modifying sidecars without redeploying services.

✗ Cons

→

Adds operational complexity by requiring deployment and management of sidecar proxies alongside services.

→

Increases resource usage since each service instance runs an additional proxy process.

→

Can introduce latency due to extra network hops through the sidecar.

Use when microservices require consistent, complex networking features like retries, circuit breaking, or mutual TLS, especially at scale with many services communicating frequently.

Avoid when the system has very few services or simple communication needs under 100 requests per second, where added complexity and resource overhead outweigh benefits.

Real World Examples

Netflix

Netflix uses sidecar proxies to implement resilient communication between microservices, enabling retries and circuit breakers without changing service code.

Google

Google's Istio service mesh uses sidecar proxies to enforce security policies and telemetry collection transparently for microservices.

Lyft

Lyft employs sidecar proxies to handle service discovery and load balancing in their microservices architecture.

Code Example

Before applying the sidecar proxy pattern, the service implements retry logic and error handling itself, mixing business and networking code. After applying the pattern, the service simply calls the local sidecar proxy endpoint. The sidecar handles retries, load balancing, and other network concerns, keeping the service code clean and focused on business logic.

Microservices

### Before: Service handles retries and logging internally
import requests

def call_service_b():
    for _ in range(3):
        try:
            response = requests.get('http://service-b/api')
            if response.status_code == 200:
                return response.json()
        except requests.exceptions.RequestException:
            pass
    raise Exception('Failed after retries')


### After: Service delegates networking to sidecar proxy
import requests

def call_service_b():
    response = requests.get('http://localhost:15001/service-b/api')  # Sidecar proxy listens locally
    response.raise_for_status()
    return response.json()

Alternatives

API Gateway

API Gateway centralizes communication at the edge, handling requests before they reach services, unlike sidecars which run alongside each service instance.

Use when: Choose API Gateway when you want centralized control over external client requests and simpler internal service communication.

Library-based networking

Networking features are implemented as libraries inside each service, unlike sidecars which are separate processes.

Use when: Choose library-based networking for simple systems with fewer services where adding sidecars is too complex.

Summary

Sidecar proxy pattern isolates networking logic from business logic by running a helper proxy alongside each service.

It centralizes retries, load balancing, and security features, improving consistency and maintainability.

This pattern is best for complex microservices architectures with high communication demands but adds operational overhead.