Request aggregation in Microservices - System Design Exercise

Design: Request Aggregation Service

Design the aggregation layer and its interaction with microservices. Out of scope: internal microservice implementations and client-side logic.

Functional Requirements

FR1: Aggregate data from multiple microservices into a single response

FR2: Support concurrent requests with low latency

FR3: Handle partial failures gracefully and return partial data when some services fail

FR4: Cache aggregated responses to improve performance for repeated requests

FR5: Provide a unified API endpoint for clients

Non-Functional Requirements

NFR1: Support up to 5000 concurrent aggregation requests

NFR2: API response time p99 under 300ms

NFR3: Availability target of 99.9% uptime

NFR4: Data freshness within 5 seconds for cached responses

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

API Gateway or Aggregation API

Service Discovery to locate microservices

Circuit Breaker for fault tolerance

Cache layer (e.g., Redis)

Load Balancer

Monitoring and Logging

Design Patterns

Aggregator pattern

Circuit Breaker pattern

Cache-Aside pattern

Bulkhead pattern

Timeout and Retry strategies

Reference Architecture

Client
  |
  v
Aggregation API (API Gateway)
  |
  |---> Cache (Redis)
  |
  |---> Microservice A
  |
  |---> Microservice B
  |
  |---> Microservice C
  |
  v
Aggregated Response

Components

Aggregation API

Node.js with Express or Spring Boot

Receives client requests, coordinates calls to microservices, aggregates responses, and returns unified data

Cache Layer

Redis

Stores recent aggregated responses to reduce latency and load on microservices

Microservices

Various (e.g., RESTful services)

Provide individual pieces of data to be aggregated

Service Discovery

Consul or Eureka

Helps Aggregation API locate microservice instances dynamically

Circuit Breaker

Resilience4j or Hystrix

Prevents cascading failures by stopping calls to failing microservices

Load Balancer

Nginx or AWS ALB

Distributes incoming requests evenly across Aggregation API instances

Request Flow

1. Client sends request to Aggregation API endpoint.

2. Aggregation API checks cache for existing aggregated response.

3. If cache hit and data is fresh, return cached response immediately.

4. If cache miss or stale data, Aggregation API concurrently calls required microservices.

5. Circuit Breaker monitors microservice calls and skips calls to failing services.

6. Aggregation API collects responses, merges data into unified format.

7. Partial data is returned if some microservices fail, with error info included.

8. Aggregated response is cached for future requests.

9. Aggregated response is sent back to client.

Database Schema

No central database required for aggregation service. Microservices maintain their own databases. Cache stores serialized aggregated responses with keys based on request parameters.

Scaling Discussion

Bottlenecks

Aggregation API CPU and memory limits under high concurrency

Cache size and eviction policy under heavy load

Latency caused by slow microservice responses

Network bandwidth between Aggregation API and microservices

Failure of multiple microservices causing degraded responses

Solutions

Scale Aggregation API horizontally with load balancer

Use distributed cache cluster with appropriate eviction policies

Implement timeouts and fallback data for slow microservices

Optimize network usage with HTTP/2 or gRPC multiplexing

Use circuit breakers and bulkheads to isolate failures and degrade gracefully

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and failure handling, 5 minutes summarizing.

Explain the need for aggregation to simplify client interactions

Discuss trade-offs between synchronous and asynchronous aggregation

Highlight fault tolerance with circuit breakers and partial responses

Describe caching strategy to improve latency and reduce load

Address scalability by horizontal scaling and caching

Mention monitoring and logging importance for production readiness

Practice

1. What is the main purpose of request aggregation in microservices?

easy

A. To cache responses from a single microservice

B. To split a large service into smaller microservices

C. To handle database transactions across services

D. To combine data from multiple microservices into a single response

Solution

1. Step 1: Understand request aggregation concept

   Request aggregation means collecting data from multiple microservices to form one combined response.

2. Step 2: Identify the main goal

   The goal is to reduce multiple client calls into one, improving efficiency and user experience.

3. Final Answer:

   To combine data from multiple microservices into a single response -> Option D

4. Quick Check:

   Request aggregation = combine multiple responses [OK]

Hint: Aggregation means combining multiple service responses [OK]

Common Mistakes:

• Confusing aggregation with service splitting
• Thinking it only caches data
• Mixing aggregation with transaction management

2. Which of the following is the correct way to implement a request aggregator in a microservices architecture?

easy

A. Make parallel calls to all required microservices and aggregate responses asynchronously

B. Make sequential calls to each microservice and combine results synchronously

C. Call only one microservice and ignore others

D. Use a database trigger to combine data from microservices

Solution

1. Step 1: Review aggregator call patterns

   Efficient aggregators call multiple services in parallel to reduce total wait time.

2. Step 2: Identify correct implementation

   Parallel asynchronous calls improve performance and user experience compared to sequential calls.

3. Final Answer:

   Make parallel calls to all required microservices and aggregate responses asynchronously -> Option A

4. Quick Check:

   Parallel async calls = best aggregator practice [OK]

Hint: Use parallel async calls for faster aggregation [OK]

Common Mistakes:

• Using sequential calls causing slow responses
• Ignoring some microservices in aggregation
• Trying to use database triggers for aggregation

3. Consider this pseudocode for a request aggregator:

async function aggregate() {
  const user = await getUser();
  const orders = await getOrders(user.id);
  const payments = await getPayments(user.id);
  return { user, orders, payments };
}

What is the main problem with this code?

medium

A. It does not handle errors from getUser

B. It calls getOrders and getPayments sequentially, increasing total response time

C. It returns data in the wrong format

D. It calls getUser multiple times unnecessarily

Solution

1. Step 1: Analyze call sequence

   The code waits for getUser, then calls getOrders and waits, then calls getPayments and waits, all sequentially.

2. Step 2: Identify inefficiency

   Calling getOrders and getPayments one after another increases total wait time unnecessarily.

3. Final Answer:

   It calls getOrders and getPayments sequentially, increasing total response time -> Option B

4. Quick Check:

   Sequential calls = slower aggregation [OK]

Hint: Parallelize independent calls to reduce wait time [OK]

Common Mistakes:

• Assuming error handling is missing
• Thinking return format is incorrect
• Believing getUser is called multiple times

4. You have a request aggregator that calls three microservices in parallel. Sometimes, one service fails and causes the whole aggregation to fail. How can you fix this?

medium

A. Cache the failed service response permanently

B. Retry the failed service indefinitely until it succeeds

C. Ignore errors and return partial data with error info for failed services

D. Stop calling other services if one fails

Solution

1. Step 1: Understand error impact in aggregation

   If one service fails, the aggregator should still return available data to avoid full failure.

2. Step 2: Choose error handling strategy

   Returning partial data with error info improves user experience and system resilience.

3. Final Answer:

   Ignore errors and return partial data with error info for failed services -> Option C

4. Quick Check:

   Partial data + error info = robust aggregation [OK]

Hint: Return partial results with errors, don't fail whole aggregation [OK]

Common Mistakes:

• Retrying endlessly causing delays
• Stopping all calls on one failure
• Caching errors permanently causing stale data

5. You design a request aggregator for a shopping app that calls user, orders, and payment microservices. To improve scalability, which design choice is best?

hard

A. Use asynchronous parallel calls with timeout and fallback data for each microservice

B. Call microservices sequentially and cache all responses for 24 hours

C. Aggregate data in a single monolithic service instead of microservices

D. Make synchronous calls and block until all microservices respond

Solution

1. Step 1: Consider scalability needs

   Parallel async calls reduce latency and improve throughput under load.

2. Step 2: Add timeout and fallback

   Timeouts prevent long waits; fallback data keeps user experience smooth if a service is slow or down.

3. Step 3: Evaluate other options

   Sequential calls and long caching reduce freshness and responsiveness; monolith loses microservices benefits; synchronous blocking hurts scalability.

4. Final Answer:

   Use asynchronous parallel calls with timeout and fallback data for each microservice -> Option A

5. Quick Check:

   Async parallel + timeout + fallback = scalable aggregator [OK]

Hint: Combine async calls with timeout and fallback for best scalability [OK]

Common Mistakes:

• Using sequential calls causing slow response
• Relying on stale cached data too long
• Ignoring microservices benefits by monolith design