Request aggregation in Microservices - System Design Exercise

Design: Request Aggregation Service

Design the aggregation layer and its interaction with microservices. Out of scope: internal microservice implementations and client-side logic.

Functional Requirements

FR1: Aggregate data from multiple microservices into a single response

FR2: Support concurrent requests with low latency

FR3: Handle partial failures gracefully and return partial data when some services fail

FR4: Cache aggregated responses to improve performance for repeated requests

FR5: Provide a unified API endpoint for clients

Non-Functional Requirements

NFR1: Support up to 5000 concurrent aggregation requests

NFR2: API response time p99 under 300ms

NFR3: Availability target of 99.9% uptime

NFR4: Data freshness within 5 seconds for cached responses

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

API Gateway or Aggregation API

Service Discovery to locate microservices

Circuit Breaker for fault tolerance

Cache layer (e.g., Redis)

Load Balancer

Monitoring and Logging

Design Patterns

Aggregator pattern

Circuit Breaker pattern

Cache-Aside pattern

Bulkhead pattern

Timeout and Retry strategies

Reference Architecture

Client
  |
  v
Aggregation API (API Gateway)
  |
  |---> Cache (Redis)
  |
  |---> Microservice A
  |
  |---> Microservice B
  |
  |---> Microservice C
  |
  v
Aggregated Response

Components

Aggregation API

Node.js with Express or Spring Boot

Receives client requests, coordinates calls to microservices, aggregates responses, and returns unified data

Cache Layer

Redis

Stores recent aggregated responses to reduce latency and load on microservices

Microservices

Various (e.g., RESTful services)

Provide individual pieces of data to be aggregated

Service Discovery

Consul or Eureka

Helps Aggregation API locate microservice instances dynamically

Circuit Breaker

Resilience4j or Hystrix

Prevents cascading failures by stopping calls to failing microservices

Load Balancer

Nginx or AWS ALB

Distributes incoming requests evenly across Aggregation API instances

Request Flow

1. Client sends request to Aggregation API endpoint.

2. Aggregation API checks cache for existing aggregated response.

3. If cache hit and data is fresh, return cached response immediately.

4. If cache miss or stale data, Aggregation API concurrently calls required microservices.

5. Circuit Breaker monitors microservice calls and skips calls to failing services.

6. Aggregation API collects responses, merges data into unified format.

7. Partial data is returned if some microservices fail, with error info included.

8. Aggregated response is cached for future requests.

9. Aggregated response is sent back to client.

Database Schema

No central database required for aggregation service. Microservices maintain their own databases. Cache stores serialized aggregated responses with keys based on request parameters.

Scaling Discussion

Bottlenecks

Aggregation API CPU and memory limits under high concurrency

Cache size and eviction policy under heavy load

Latency caused by slow microservice responses

Network bandwidth between Aggregation API and microservices

Failure of multiple microservices causing degraded responses

Solutions

Scale Aggregation API horizontally with load balancer

Use distributed cache cluster with appropriate eviction policies

Implement timeouts and fallback data for slow microservices

Optimize network usage with HTTP/2 or gRPC multiplexing

Use circuit breakers and bulkheads to isolate failures and degrade gracefully

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and failure handling, 5 minutes summarizing.

Explain the need for aggregation to simplify client interactions

Discuss trade-offs between synchronous and asynchronous aggregation

Highlight fault tolerance with circuit breakers and partial responses

Describe caching strategy to improve latency and reduce load

Address scalability by horizontal scaling and caching

Mention monitoring and logging importance for production readiness