Ambassador pattern in Microservices - Scalability & System Analysis

Scalability Analysis - Ambassador pattern

Growth Table: Ambassador Pattern Scaling

Users/Traffic	What Changes
100 users	Single ambassador instance per service; low network overhead; simple routing and monitoring.
10,000 users	Multiple ambassador instances per service; increased network traffic; ambassador handles retries, circuit breaking; load balancing needed.
1,000,000 users	Ambassador instances scale horizontally; network bandwidth grows; ambassador becomes critical for service discovery and security; monitoring and logging volume increases.
100,000,000 users	Massive horizontal scaling of ambassadors; potential network bottlenecks; need for ambassador clustering or federation; advanced caching and rate limiting; observability systems must scale.

First Bottleneck

The first bottleneck is the ambassador proxy instances themselves. As traffic grows, each ambassador handles more connections and network processing. CPU and memory limits on ambassador containers or VMs cause latency and throughput issues before backend services do.

Scaling Solutions

• Horizontal scaling: Add more ambassador instances per service to distribute load.
• Load balancing: Use external load balancers or service mesh features to balance traffic among ambassadors.
• Caching: Implement response caching in ambassadors to reduce backend calls.
• Connection pooling: Reuse connections to backend services to reduce overhead.
• Rate limiting and circuit breaking: Prevent overload and isolate failures at ambassador level.
• Monitoring and autoscaling: Use metrics to trigger scaling of ambassador pods or instances.
• Network optimization: Use efficient protocols and compression to reduce bandwidth.

Back-of-Envelope Cost Analysis

• Each ambassador handles ~1000-5000 concurrent connections.
• At 1 million users, assuming 10 requests per second per user, total requests = 10 million QPS.
• To handle 10 million QPS, need ~2000 ambassador instances (assuming 5000 QPS per instance).
• Network bandwidth: 1 Gbps = 125 MB/s; high traffic requires multiple 10 Gbps links or cloud bandwidth scaling.
• Storage for logs and metrics grows with traffic; plan for scalable observability backend.

Interview Tip

Start by explaining the ambassador pattern role as a proxy for microservices. Discuss how it helps with cross-cutting concerns like retries and monitoring. Then analyze scaling by identifying the ambassador as the first bottleneck. Propose horizontal scaling and caching as solutions. Finally, mention observability and network considerations to show a full picture.

Self Check

Your ambassador proxy handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add more ambassador instances to horizontally scale and distribute the increased load, ensuring each instance stays within its capacity.

Key Result

The ambassador proxy is the first bottleneck as traffic grows; horizontal scaling and caching at the ambassador layer are key to maintaining performance.

Practice

1. What is the main purpose of the Ambassador pattern in microservices architecture?

easy

A. To directly expose services to the internet without any proxy

B. To replace the main service with a new version

C. To store data in a centralized database

D. To add a helper component that manages communication between services

Solution

1. Step 1: Understand the role of the Ambassador pattern

   The Ambassador pattern introduces a helper component that acts as a proxy or sidecar to handle communication tasks for a service.

2. Step 2: Compare options with the pattern's purpose

   Replacing services, storing data centrally, or exposing services directly do not describe the Ambassador pattern's role.

3. Final Answer:

   To add a helper component that manages communication between services -> Option D

4. Quick Check:

   Ambassador pattern = helper component for communication [OK]

Hint: Ambassador adds a helper proxy for communication [OK]

Common Mistakes:

• Confusing Ambassador with database or service replacement
• Thinking it exposes services directly without proxy
• Assuming it stores data centrally

2. Which of the following is the correct way to describe the Ambassador pattern's deployment style?

easy

A. A sidecar proxy deployed alongside the main service

B. A centralized database for service communication

C. A standalone service that replaces the main service

D. A load balancer that distributes traffic among services

Solution

1. Step 1: Identify deployment style of Ambassador pattern

   The Ambassador pattern is typically deployed as a sidecar proxy next to the main service to handle communication.

2. Step 2: Eliminate incorrect deployment types

   It is not a standalone replacement, centralized database, or load balancer.

3. Final Answer:

   A sidecar proxy deployed alongside the main service -> Option A

4. Quick Check:

   Ambassador deployment = sidecar proxy [OK]

Hint: Ambassador runs as a sidecar proxy next to service [OK]

Common Mistakes:

• Thinking Ambassador replaces the main service
• Confusing it with load balancer or database
• Assuming it is a standalone service

3. Consider this simplified pseudo-code for an Ambassador proxy handling requests:

class AmbassadorProxy {
  sendRequest(request) {
    if (this.isServiceAvailable()) {
      return this.forward(request);
    } else {
      return this.retry(request);
    }
  }
}

What will happen if the main service is temporarily down?

medium

A. The proxy forwards the request without checking availability

B. The proxy retries sending the request until the service is available

C. The proxy immediately returns an error without retrying

D. The proxy stores the request permanently without forwarding

Solution

1. Step 1: Analyze the sendRequest method logic

   The method checks if the service is available. If yes, it forwards the request; otherwise, it retries.

2. Step 2: Determine behavior when service is down

   If the service is down, isServiceAvailable() returns false, so retry(request) is called to resend the request.

3. Final Answer:

   The proxy retries sending the request until the service is available -> Option B

4. Quick Check:

   Service down triggers retry in Ambassador proxy [OK]

Hint: Ambassador retries requests if service unavailable [OK]

Common Mistakes:

• Assuming proxy forwards without checking
• Thinking proxy returns error immediately
• Believing proxy stores requests permanently

4. A developer wrote this Ambassador proxy code snippet:

class Ambassador {
  send(request) {
    if (this.checkService()) {
      this.forward(request);
    } else {
      this.retry(request);
    }
  }
}

But requests are never retried even when the service is down. What is the likely bug?

medium

A. The send method does not return the result of retry or forward

B. The method forward does not return the response

C. The checkService method always returns true

D. The retry method is not implemented

Solution

1. Step 1: Review send method behavior

   The send method calls forward or retry but does not return their results, so caller never sees retries.

2. Step 2: Understand impact on retry behavior

   Without returning retry's result, retries may happen but caller ignores them, appearing as if no retry occurs.

3. Final Answer:

   The send method does not return the result of retry or forward -> Option A

4. Quick Check:

   Missing return in send causes retry to be ignored [OK]

Hint: Always return retry/forward results in proxy methods [OK]

Common Mistakes:

• Assuming checkService always true without checking
• Thinking retry method is missing
• Ignoring return values in send method

5. You want to improve observability and security for a microservice without changing its code. How does the Ambassador pattern help achieve this?

hard

A. By replacing the microservice with a new secure version

B. By modifying the service code to include security and logging libraries

C. By adding a sidecar proxy that handles logging, retries, and TLS encryption transparently

D. By deploying a centralized monitoring service that polls the microservice directly

Solution

1. Step 1: Identify how Ambassador pattern enhances observability and security

   The Ambassador pattern uses a sidecar proxy to add features like logging, retries, and TLS without changing the main service code.

2. Step 2: Compare with other options

   Modifying service code, polling directly, or replacing service do not align with Ambassador pattern benefits.

3. Final Answer:

   By adding a sidecar proxy that handles logging, retries, and TLS encryption transparently -> Option C

4. Quick Check:

   Ambassador adds security and observability via sidecar proxy [OK]

Hint: Ambassador adds features without changing service code [OK]

Common Mistakes:

• Thinking service code must be changed
• Confusing Ambassador with centralized monitoring
• Assuming service replacement is needed