gRPC for internal communication in Microservices - System Design Exercise

Design: Microservices Internal Communication using gRPC

Design focuses on internal communication between microservices using gRPC. Out of scope are external client communication, UI design, and persistent storage details.

Functional Requirements

FR1: Enable efficient communication between multiple microservices within the same infrastructure

FR2: Support synchronous request-response calls with low latency

FR3: Allow definition of clear service contracts with strong typing

FR4: Support multiple programming languages for microservices

FR5: Ensure secure communication between services

FR6: Handle service discovery and load balancing internally

FR7: Provide error handling and retries for transient failures

Non-Functional Requirements

NFR1: Must handle up to 10,000 requests per second across services

NFR2: End-to-end latency for internal calls should be under 50ms p99

NFR3: Availability target of 99.9% uptime for internal communication

NFR4: Use existing infrastructure without adding heavy new components

NFR5: Minimize network overhead and serialization costs

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

gRPC server and client stubs generated from protobuf definitions

Service registry or discovery mechanism

Load balancer for distributing requests

TLS for secure communication

Monitoring and logging tools for tracing calls

Retry and timeout configurations

Design Patterns

API Gateway pattern for routing

Circuit Breaker pattern for fault tolerance

Client-side load balancing

Service mesh integration for observability and security

Protobuf schema versioning for backward compatibility

Reference Architecture

  +----------------+       +----------------+       +----------------+
  |  Microservice  | <---> |  gRPC Server   | <---> |  Microservice  |
  |     A          |       |  (Service B)   |       |     B          |
  +----------------+       +----------------+       +----------------+
          |                        |                        |
          | gRPC Client            | gRPC Server            | gRPC Client
          |                        |                        |
  +---------------------------------------------------------------+
  |                    Service Discovery & Load Balancer          |
  +---------------------------------------------------------------+
                              |
                              | TLS Secured gRPC Communication
                              |
                      +------------------+
                      | Monitoring & Logs |
                      +------------------+

Components

gRPC Server

gRPC with Protocol Buffers

Expose service APIs with strongly typed contracts for other microservices to call

gRPC Client

gRPC with Protocol Buffers

Invoke remote service APIs efficiently with low latency

Service Discovery

Consul / etcd / Kubernetes DNS

Enable clients to find available service instances dynamically

Load Balancer

Client-side or Envoy proxy

Distribute requests evenly across service instances

TLS Encryption

mTLS (mutual TLS)

Secure communication between microservices

Monitoring & Logging

Prometheus, Jaeger, Fluentd

Trace requests, monitor latency, and log errors

Request Flow

1. 1. Microservice A wants to call Microservice B's API.

2. 2. Microservice A's gRPC client queries the service discovery to get available instances of Microservice B.

3. 3. The client selects an instance using load balancing strategy.

4. 4. The client establishes a secure (mTLS) gRPC connection to Microservice B's gRPC server.

5. 5. Microservice A sends a request message defined by protobuf to Microservice B.

6. 6. Microservice B processes the request and sends back a response message.

7. 7. Microservice A receives the response and continues processing.

8. 8. Monitoring tools collect metrics and traces for this call.

Database Schema

Not applicable as this design focuses on communication protocols and infrastructure rather than data storage.

Scaling Discussion

Bottlenecks

Service discovery becoming a single point of failure under high load

Load balancer overwhelmed by large number of requests

Network bandwidth limits causing latency spikes

TLS handshake overhead impacting latency

gRPC server CPU or memory saturation

Solutions

Use highly available and distributed service discovery systems with caching

Implement client-side load balancing to reduce central load balancer pressure

Optimize network infrastructure and use compression for gRPC messages

Reuse TLS connections and enable session resumption to reduce handshake cost

Scale out gRPC servers horizontally and use autoscaling based on metrics

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, and 5 minutes summarizing.

Explain why gRPC is suitable for internal microservice communication (performance, strong typing)

Discuss how service discovery and load balancing work together

Highlight security with mTLS for internal calls

Describe monitoring and observability importance

Address scaling challenges and practical solutions