Ambassador pattern in Microservices - System Design Exercise

Design: Microservices Communication with Ambassador Pattern

Design focuses on the ambassador pattern as a proxy sidecar for microservices to external services. It excludes internal microservice business logic and database design.

Functional Requirements

FR1: Enable microservices to communicate with external services securely and reliably

FR2: Handle retries, timeouts, and circuit breaking for external calls

FR3: Allow easy configuration and updates without changing microservice code

FR4: Support logging and monitoring of external service calls

FR5: Ensure minimal latency overhead for service-to-service communication

Non-Functional Requirements

NFR1: System must handle 10,000 requests per second

NFR2: API response latency p99 should be under 300ms including ambassador overhead

NFR3: Availability target of 99.9% uptime

NFR4: Must support multiple external services with different protocols

NFR5: Deployment should allow independent scaling of ambassador components

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Ambassador proxy component (sidecar or standalone)

Microservice application

External services (APIs, databases, third-party services)

Configuration management system

Monitoring and logging tools

Design Patterns

Sidecar pattern

Circuit breaker pattern

Retry pattern

Proxy pattern

Service mesh integration

Reference Architecture

  +----------------+       +----------------+       +--------------------+
  |                |       |                |       |                    |
  |  Microservice 1|<----->| Ambassador 1   |<----->| External Service A  |
  |                |       | (Sidecar Proxy)|       |                    |
  +----------------+       +----------------+       +--------------------+

  +----------------+       +----------------+       +--------------------+
  |                |       |                |       |                    |
  |  Microservice 2|<----->| Ambassador 2   |<----->| External Service B  |
  |                |       | (Sidecar Proxy)|       |                    |
  +----------------+       +----------------+       +--------------------+

Components

Microservice

Any microservice framework (e.g., Spring Boot, Node.js, Go)

Business logic that needs to call external services

Ambassador Proxy

Envoy, NGINX, or custom proxy sidecar

Handles all external service communication, retries, timeouts, circuit breaking, and logging

External Services

REST APIs, gRPC services, databases, third-party APIs

Services outside the microservice boundary that need to be accessed

Configuration Management

Consul, etcd, or Kubernetes ConfigMaps

Manage ambassador proxy configurations dynamically

Monitoring and Logging

Prometheus, Grafana, ELK stack

Collect metrics and logs from ambassador proxies for observability

Request Flow

1. 1. Microservice sends a request to the Ambassador proxy (sidecar) instead of directly calling the external service.

2. 2. Ambassador proxy applies retry policies, timeout settings, and circuit breaker logic before forwarding the request.

3. 3. Ambassador proxy sends the request to the external service.

4. 4. External service processes the request and sends a response back to the Ambassador proxy.

5. 5. Ambassador proxy logs the request and response details, updates metrics, and returns the response to the microservice.

6. 6. Configuration changes for the Ambassador proxy can be applied dynamically without restarting the microservice.

Database Schema

Not applicable as the ambassador pattern focuses on communication proxies and does not require a database schema.

Scaling Discussion

Bottlenecks

Ambassador proxy becomes a bottleneck under high request volume

Configuration management delays causing outdated proxy settings

Monitoring overhead impacting proxy performance

Network latency between microservice, ambassador, and external services

Solutions

Deploy multiple ambassador instances per microservice and use load balancing

Use a highly available and fast configuration store with push-based updates

Optimize monitoring to sample or aggregate metrics to reduce overhead

Place ambassador proxies close to microservices and external services to reduce latency

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing the ambassador pattern architecture and data flow, 10 minutes discussing scaling and trade-offs, and 5 minutes summarizing.

Explain how the ambassador pattern acts as a sidecar proxy to handle external service calls

Discuss benefits like separation of concerns, easier retries, and centralized logging

Describe how configuration can be managed dynamically

Highlight how this pattern improves reliability and observability

Address scaling challenges and solutions clearly