Ambassador pattern in Microservices - System Design Exercise

Design: Microservices Communication with Ambassador Pattern

Design focuses on the ambassador pattern as a proxy sidecar for microservices to external services. It excludes internal microservice business logic and database design.

Functional Requirements

FR1: Enable microservices to communicate with external services securely and reliably

FR2: Handle retries, timeouts, and circuit breaking for external calls

FR3: Allow easy configuration and updates without changing microservice code

FR4: Support logging and monitoring of external service calls

FR5: Ensure minimal latency overhead for service-to-service communication

Non-Functional Requirements

NFR1: System must handle 10,000 requests per second

NFR2: API response latency p99 should be under 300ms including ambassador overhead

NFR3: Availability target of 99.9% uptime

NFR4: Must support multiple external services with different protocols

NFR5: Deployment should allow independent scaling of ambassador components

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Ambassador proxy component (sidecar or standalone)

Microservice application

External services (APIs, databases, third-party services)

Configuration management system

Monitoring and logging tools

Design Patterns

Sidecar pattern

Circuit breaker pattern

Retry pattern

Proxy pattern

Service mesh integration

Reference Architecture

  +----------------+       +----------------+       +--------------------+
  |                |       |                |       |                    |
  |  Microservice 1|<----->| Ambassador 1   |<----->| External Service A  |
  |                |       | (Sidecar Proxy)|       |                    |
  +----------------+       +----------------+       +--------------------+

  +----------------+       +----------------+       +--------------------+
  |                |       |                |       |                    |
  |  Microservice 2|<----->| Ambassador 2   |<----->| External Service B  |
  |                |       | (Sidecar Proxy)|       |                    |
  +----------------+       +----------------+       +--------------------+

Components

Microservice

Any microservice framework (e.g., Spring Boot, Node.js, Go)

Business logic that needs to call external services

Ambassador Proxy

Envoy, NGINX, or custom proxy sidecar

Handles all external service communication, retries, timeouts, circuit breaking, and logging

External Services

REST APIs, gRPC services, databases, third-party APIs

Services outside the microservice boundary that need to be accessed

Configuration Management

Consul, etcd, or Kubernetes ConfigMaps

Manage ambassador proxy configurations dynamically

Monitoring and Logging

Prometheus, Grafana, ELK stack

Collect metrics and logs from ambassador proxies for observability

Request Flow

1. 1. Microservice sends a request to the Ambassador proxy (sidecar) instead of directly calling the external service.

2. 2. Ambassador proxy applies retry policies, timeout settings, and circuit breaker logic before forwarding the request.

3. 3. Ambassador proxy sends the request to the external service.

4. 4. External service processes the request and sends a response back to the Ambassador proxy.

5. 5. Ambassador proxy logs the request and response details, updates metrics, and returns the response to the microservice.

6. 6. Configuration changes for the Ambassador proxy can be applied dynamically without restarting the microservice.

Database Schema

Not applicable as the ambassador pattern focuses on communication proxies and does not require a database schema.

Scaling Discussion

Bottlenecks

Ambassador proxy becomes a bottleneck under high request volume

Configuration management delays causing outdated proxy settings

Monitoring overhead impacting proxy performance

Network latency between microservice, ambassador, and external services

Solutions

Deploy multiple ambassador instances per microservice and use load balancing

Use a highly available and fast configuration store with push-based updates

Optimize monitoring to sample or aggregate metrics to reduce overhead

Place ambassador proxies close to microservices and external services to reduce latency

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing the ambassador pattern architecture and data flow, 10 minutes discussing scaling and trade-offs, and 5 minutes summarizing.

Explain how the ambassador pattern acts as a sidecar proxy to handle external service calls

Discuss benefits like separation of concerns, easier retries, and centralized logging

Describe how configuration can be managed dynamically

Highlight how this pattern improves reliability and observability

Address scaling challenges and solutions clearly

Practice

1. What is the main purpose of the Ambassador pattern in microservices architecture?

easy

A. To directly expose services to the internet without any proxy

B. To replace the main service with a new version

C. To store data in a centralized database

D. To add a helper component that manages communication between services

Solution

1. Step 1: Understand the role of the Ambassador pattern

   The Ambassador pattern introduces a helper component that acts as a proxy or sidecar to handle communication tasks for a service.

2. Step 2: Compare options with the pattern's purpose

   Replacing services, storing data centrally, or exposing services directly do not describe the Ambassador pattern's role.

3. Final Answer:

   To add a helper component that manages communication between services -> Option D

4. Quick Check:

   Ambassador pattern = helper component for communication [OK]

Hint: Ambassador adds a helper proxy for communication [OK]

Common Mistakes:

• Confusing Ambassador with database or service replacement
• Thinking it exposes services directly without proxy
• Assuming it stores data centrally

2. Which of the following is the correct way to describe the Ambassador pattern's deployment style?

easy

A. A sidecar proxy deployed alongside the main service

B. A centralized database for service communication

C. A standalone service that replaces the main service

D. A load balancer that distributes traffic among services

Solution

1. Step 1: Identify deployment style of Ambassador pattern

   The Ambassador pattern is typically deployed as a sidecar proxy next to the main service to handle communication.

2. Step 2: Eliminate incorrect deployment types

   It is not a standalone replacement, centralized database, or load balancer.

3. Final Answer:

   A sidecar proxy deployed alongside the main service -> Option A

4. Quick Check:

   Ambassador deployment = sidecar proxy [OK]

Hint: Ambassador runs as a sidecar proxy next to service [OK]

Common Mistakes:

• Thinking Ambassador replaces the main service
• Confusing it with load balancer or database
• Assuming it is a standalone service

3. Consider this simplified pseudo-code for an Ambassador proxy handling requests:

class AmbassadorProxy {
  sendRequest(request) {
    if (this.isServiceAvailable()) {
      return this.forward(request);
    } else {
      return this.retry(request);
    }
  }
}

What will happen if the main service is temporarily down?

medium

A. The proxy forwards the request without checking availability

B. The proxy retries sending the request until the service is available

C. The proxy immediately returns an error without retrying

D. The proxy stores the request permanently without forwarding

Solution

1. Step 1: Analyze the sendRequest method logic

   The method checks if the service is available. If yes, it forwards the request; otherwise, it retries.

2. Step 2: Determine behavior when service is down

   If the service is down, isServiceAvailable() returns false, so retry(request) is called to resend the request.

3. Final Answer:

   The proxy retries sending the request until the service is available -> Option B

4. Quick Check:

   Service down triggers retry in Ambassador proxy [OK]

Hint: Ambassador retries requests if service unavailable [OK]

Common Mistakes:

• Assuming proxy forwards without checking
• Thinking proxy returns error immediately
• Believing proxy stores requests permanently

4. A developer wrote this Ambassador proxy code snippet:

class Ambassador {
  send(request) {
    if (this.checkService()) {
      this.forward(request);
    } else {
      this.retry(request);
    }
  }
}

But requests are never retried even when the service is down. What is the likely bug?

medium

A. The send method does not return the result of retry or forward

B. The method forward does not return the response

C. The checkService method always returns true

D. The retry method is not implemented

Solution

1. Step 1: Review send method behavior

   The send method calls forward or retry but does not return their results, so caller never sees retries.

2. Step 2: Understand impact on retry behavior

   Without returning retry's result, retries may happen but caller ignores them, appearing as if no retry occurs.

3. Final Answer:

   The send method does not return the result of retry or forward -> Option A

4. Quick Check:

   Missing return in send causes retry to be ignored [OK]

Hint: Always return retry/forward results in proxy methods [OK]

Common Mistakes:

• Assuming checkService always true without checking
• Thinking retry method is missing
• Ignoring return values in send method

5. You want to improve observability and security for a microservice without changing its code. How does the Ambassador pattern help achieve this?

hard

A. By replacing the microservice with a new secure version

B. By modifying the service code to include security and logging libraries

C. By adding a sidecar proxy that handles logging, retries, and TLS encryption transparently

D. By deploying a centralized monitoring service that polls the microservice directly

Solution

1. Step 1: Identify how Ambassador pattern enhances observability and security

   The Ambassador pattern uses a sidecar proxy to add features like logging, retries, and TLS without changing the main service code.

2. Step 2: Compare with other options

   Modifying service code, polling directly, or replacing service do not align with Ambassador pattern benefits.

3. Final Answer:

   By adding a sidecar proxy that handles logging, retries, and TLS encryption transparently -> Option C

4. Quick Check:

   Ambassador adds security and observability via sidecar proxy [OK]

Hint: Ambassador adds features without changing service code [OK]

Common Mistakes:

• Thinking service code must be changed
• Confusing Ambassador with centralized monitoring
• Assuming service replacement is needed