gRPC for internal services in HLD - System Design Exercise

Design: Internal Microservices Communication using gRPC

Design focuses on internal service-to-service communication using gRPC. Out of scope are external client communication, persistent storage design, and UI components.

Functional Requirements

FR1: Enable efficient communication between multiple internal microservices

FR2: Support synchronous request-response calls with low latency

FR3: Allow easy definition and evolution of service APIs

FR4: Ensure strong typing and contract enforcement between services

FR5: Support load balancing and service discovery within the internal network

FR6: Provide mechanisms for error handling and retries

FR7: Secure communication within the internal network

Non-Functional Requirements

NFR1: Handle up to 10,000 requests per second across services

NFR2: Maintain p99 latency under 50ms for RPC calls

NFR3: Ensure 99.9% availability for internal service communication

NFR4: Use protobuf for message serialization

NFR5: Operate within a trusted internal network environment

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

gRPC client and server stubs generated from protobuf definitions

Service registry or discovery mechanism (e.g., DNS, Consul, or Kubernetes)

Load balancer for distributing requests

Authentication and authorization middleware

Monitoring and logging tools for RPC calls

Retry and timeout policies

Design Patterns

API contract-first design with protobuf

Unary RPC vs streaming RPC patterns

Client-side vs server-side load balancing

Circuit breaker and retry patterns

Versioning strategies for protobuf APIs

Reference Architecture

                    +---------------------+
                    |   Service Registry   |
                    +----------+----------+
                               |
          +--------------------+--------------------+
          |                                         |
  +-------v-------+                         +-------v-------+
  |  Microservice |                         |  Microservice |
  |     A (gRPC)  |                         |     B (gRPC)  |
  +-------+-------+                         +-------+-------+
          |                                         |
          +--------------------+--------------------+
                               |
                    +----------v----------+
                    |    Load Balancer    |
                    +---------------------+

Components

gRPC Server Stub

gRPC with Protobuf

Defines and implements service APIs to handle incoming RPC calls.

gRPC Client Stub

gRPC with Protobuf

Allows microservices to call other services using generated client code.

Service Registry

Consul / Kubernetes DNS

Keeps track of available service instances and their network locations.

Load Balancer

Envoy / gRPC built-in load balancing

Distributes requests evenly across healthy service instances.

Authentication Middleware

mTLS / JWT

Secures communication between services within the internal network.

Monitoring and Logging

Prometheus / Grafana / OpenTelemetry

Tracks RPC call metrics, latency, errors for observability.

Request Flow

1. 1. Microservice A wants to call Microservice B's API.

2. 2. Microservice A uses the gRPC client stub generated from protobuf to create a request.

3. 3. The client queries the Service Registry to find available instances of Microservice B.

4. 4. The Load Balancer selects a healthy instance of Microservice B.

5. 5. The request is sent over HTTP/2 with protobuf serialization using gRPC protocol.

6. 6. Authentication middleware verifies the identity and permissions of Microservice A.

7. 7. Microservice B's gRPC server stub receives the request and processes it.

8. 8. Microservice B sends back the response via gRPC over HTTP/2.

9. 9. Microservice A receives the response and continues processing.

Database Schema

Not applicable as this design focuses on communication between services, not data storage.

Scaling Discussion

Bottlenecks

Service Registry becoming a single point of failure or bottleneck.

Load Balancer overload under high request volume.

Network latency and bandwidth limits affecting RPC call performance.

Large message sizes causing serialization/deserialization delays.

Version incompatibility between services causing communication failures.

Solutions

Use highly available and distributed service registries like Consul or Kubernetes DNS.

Implement client-side load balancing to reduce dependency on central load balancers.

Optimize network infrastructure and use compression for messages.

Limit message sizes and use streaming RPC for large data transfers.

Adopt strict API versioning and backward compatibility strategies.

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, and 5 minutes summarizing.

Explain why gRPC is suitable for internal microservices communication (low latency, strong typing).

Discuss protobuf for API contract and versioning benefits.

Highlight service discovery and load balancing importance.

Mention security considerations even within internal networks.

Address how to handle failures and retries gracefully.

Talk about monitoring for observability and debugging.