gRPC for internal services in HLD - Scalability & System Analysis

Scalability Analysis - gRPC for internal services

Growth Table: gRPC for Internal Services

Scale	Users / Services	Traffic Characteristics	Infrastructure Changes	Latency & Throughput
100 users	5-10 internal services	Low QPS, mostly request-response	Single server or small cluster, simple load balancing	Low latency (~ms), high throughput easily handled
10K users	20-50 internal services	Moderate QPS, some streaming calls	Multiple servers, load balancers, basic service discovery	Latency remains low, throughput increases, some resource contention
1M users	100+ internal services	High QPS, mix of unary and streaming, complex call graphs	Horizontal scaling, advanced service mesh, distributed tracing	Latency sensitive, throughput near server limits, network bottlenecks appear
100M users	1000+ internal services	Very high QPS, heavy streaming, multi-region calls	Global clusters, sharded services, aggressive caching, CDN for static data	Latency optimization critical, throughput requires sharding and partitioning

First Bottleneck

At small to medium scale, the first bottleneck is the application server CPU and network. gRPC uses HTTP/2 which multiplexes streams but high QPS and streaming calls can saturate CPU and network bandwidth on servers.

At larger scale, the service discovery and load balancing become bottlenecks as the number of services and calls grow. Also, network latency and bandwidth between services in different regions can limit performance.

Scaling Solutions

• Horizontal scaling: Add more instances of services behind load balancers to distribute load.
• Service mesh: Use tools like Istio or Linkerd for advanced routing, retries, and observability.
• Caching: Cache frequent responses to reduce load on services.
• Connection pooling: Reuse gRPC connections to reduce overhead.
• Sharding: Partition services or data to reduce load per instance.
• Compression: Enable gRPC message compression to reduce network usage.
• Multi-region deployment: Deploy services closer to users to reduce latency.

Back-of-Envelope Cost Analysis

Assuming 1M users generating 10 QPS each internally (10M QPS total):

• Each server handles ~3000 concurrent gRPC streams.
• Need ~3333 servers to handle 10M QPS (10M / 3000 ≈ 3333).
• Network bandwidth per server: If average message size is 10KB, then 3000 * 10KB = ~30MB/s (~240Mbps), within 1Gbps NIC capacity.
• Storage depends on logging and tracing; distributed tracing data can be large and needs separate storage.

Interview Tip

Start by clarifying the scale and traffic patterns. Identify the first bottleneck based on expected QPS and message sizes. Discuss horizontal scaling and service mesh for routing and observability. Mention connection reuse and caching to optimize performance. Finally, consider multi-region deployment for latency-sensitive services.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Add read replicas and implement caching to reduce load on the primary database before scaling application servers.

Key Result

gRPC scales well with horizontal server scaling and service mesh support, but CPU, network bandwidth, and service discovery become bottlenecks at high QPS; solutions include connection pooling, caching, sharding, and multi-region deployment.

Practice

1. What is the main advantage of using gRPC for communication between internal services?

easy

A. It provides fast and efficient communication using HTTP/2 and Protocol Buffers.

B. It uses plain text JSON for easy debugging.

C. It requires no schema definition for messages.

D. It only works for external client-server communication.

Solution

1. Step 1: Understand gRPC communication basics

   gRPC uses HTTP/2 for transport and Protocol Buffers for message serialization, making it fast and efficient.

2. Step 2: Compare options with gRPC features

   Options A and B are incorrect because gRPC uses binary Protocol Buffers, not JSON, and requires schema definitions. It only works for external client-server communication. is wrong because gRPC is often used internally.

3. Final Answer:

   It provides fast and efficient communication using HTTP/2 and Protocol Buffers. -> Option A

4. Quick Check:

   gRPC speed and efficiency = It provides fast and efficient communication using HTTP/2 and Protocol Buffers. [OK]

Hint: Remember gRPC uses HTTP/2 + Protocol Buffers for speed [OK]

Common Mistakes:

• Confusing gRPC with REST using JSON
• Thinking gRPC doesn't need schemas
• Assuming gRPC is only for external APIs

2. Which of the following is the correct way to define a gRPC service method in a Protocol Buffers (.proto) file?

easy

A. service UserService { function GetUser (UserRequest) returns UserResponse; }

B. service UserService { call GetUser (UserRequest) returns UserResponse; }

C. service UserService { method GetUser (UserRequest) -> UserResponse; }

D. service UserService { rpc GetUser (UserRequest) returns (UserResponse); }

Solution

1. Step 1: Recall gRPC service syntax in .proto files

   The correct syntax uses rpc MethodName(RequestType) returns (ResponseType); inside a service block.

2. Step 2: Match options to correct syntax

   Only service UserService { rpc GetUser (UserRequest) returns (UserResponse); } matches the exact gRPC syntax. Others use incorrect keywords like function, method, or call.

3. Final Answer:

   service UserService { rpc GetUser (UserRequest) returns (UserResponse); } -> Option D

4. Quick Check:

   gRPC method syntax = service UserService { rpc GetUser (UserRequest) returns (UserResponse); } [OK]

Hint: Look for 'rpc' keyword inside service block [OK]

Common Mistakes:

• Using 'function' or 'method' instead of 'rpc'
• Omitting parentheses around request/response types
• Misplacing 'returns' keyword

3. Given this gRPC service definition snippet:

service OrderService {
  rpc CreateOrder (OrderRequest) returns (OrderResponse);
}

What happens when a client calls CreateOrder with invalid data that violates the schema?

medium

A. The server returns a validation error before processing the request.

B. The server accepts the request and processes it anyway.

C. The client call fails at compile time due to schema mismatch.

D. The server crashes due to invalid input.

Solution

1. Step 1: Understand gRPC schema validation

   gRPC uses Protocol Buffers which enforce message structure. Invalid data is rejected by the server with an error.

2. Step 2: Evaluate options based on gRPC behavior

   The server returns a validation error before processing the request. is correct because the server validates input and returns errors. The server accepts the request and processes it anyway. is unsafe and incorrect. The client call fails at compile time due to schema mismatch. is wrong because client code compiles but runtime validation happens. The server crashes due to invalid input. is incorrect as servers handle errors gracefully.

3. Final Answer:

   The server returns a validation error before processing the request. -> Option A

4. Quick Check:

   gRPC input validation = The server returns a validation error before processing the request. [OK]

Hint: Invalid data triggers server validation error, not crash [OK]

Common Mistakes:

• Assuming client compile-time failure for invalid data
• Thinking server processes invalid requests
• Believing server crashes on bad input

4. You notice that your internal gRPC service calls are slow and sometimes timeout. Which of the following is the best first step to debug this issue?

medium

A. Remove Protocol Buffers and switch to JSON messages.

B. Rewrite all services to use REST instead of gRPC.

C. Check network latency and HTTP/2 connection health between services.

D. Increase client timeout without investigating.

Solution

1. Step 1: Identify common causes of gRPC slowness

   Network issues and HTTP/2 connection problems often cause delays in gRPC calls.

2. Step 2: Evaluate debugging approaches

   Check network latency and HTTP/2 connection health between services. is the best first step to diagnose root cause. Options B and C are drastic changes without diagnosis. Increase client timeout without investigating. ignores the problem source.

3. Final Answer:

   Check network latency and HTTP/2 connection health between services. -> Option C

4. Quick Check:

   Debug gRPC slowness = Check network latency and HTTP/2 connection health between services. [OK]

Hint: Start debugging network and HTTP/2 before changing code [OK]

Common Mistakes:

• Jumping to rewrite services without diagnosis
• Ignoring network as a cause
• Blindly increasing timeouts

5. You are designing an internal microservices system using gRPC. Which design choice best improves scalability and fault isolation?

hard

A. Avoid using Protocol Buffers and send raw JSON over gRPC.

B. Define small, focused gRPC services with clear method boundaries and use load balancing.

C. Use gRPC only for external APIs and REST internally.

D. Combine all functionality into one large gRPC service to reduce network calls.

Solution

1. Step 1: Understand microservices design principles with gRPC

   Small, focused services with clear APIs improve scalability and fault isolation. Load balancing helps distribute load.

2. Step 2: Evaluate design options

   Define small, focused gRPC services with clear method boundaries and use load balancing. aligns with best practices. Combine all functionality into one large gRPC service to reduce network calls. creates a monolith, hurting scalability. Use gRPC only for external APIs and REST internally. wastes gRPC benefits internally. Avoid using Protocol Buffers and send raw JSON over gRPC. breaks gRPC efficiency by avoiding Protocol Buffers.

3. Final Answer:

   Define small, focused gRPC services with clear method boundaries and use load balancing. -> Option B

4. Quick Check:

   Microservices + gRPC best practice = Define small, focused gRPC services with clear method boundaries and use load balancing. [OK]

Hint: Small services + load balancing = scalable gRPC design [OK]

Common Mistakes:

• Building large monolithic gRPC services
• Using REST internally instead of gRPC
• Ignoring Protocol Buffers benefits