Services and networking in Microservices - System Design Exercise

Design: Microservices Communication and Networking System

Design focuses on networking and communication between microservices including discovery, load balancing, security, and monitoring. Does not cover internal service business logic or database design.

Functional Requirements

FR1: Support multiple microservices communicating with each other

FR2: Enable service discovery so services can find each other dynamically

FR3: Provide load balancing for requests across service instances

FR4: Ensure secure communication between services

FR5: Handle failures gracefully with retries and circuit breakers

FR6: Allow monitoring and tracing of service calls

Non-Functional Requirements

NFR1: Must support up to 10,000 concurrent service-to-service requests

NFR2: API response latency p99 under 200ms for inter-service calls

NFR3: Availability target of 99.9% uptime for service communication

NFR4: Support dynamic scaling of services without manual reconfiguration

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Service registry for discovery

API gateway or service mesh for routing and load balancing

Secure communication protocols (TLS)

Circuit breaker and retry mechanisms

Distributed tracing system

Monitoring and alerting tools

Design Patterns

Service discovery pattern

Client-side vs server-side load balancing

Circuit breaker pattern

Sidecar proxy pattern

Distributed tracing and correlation IDs

Reference Architecture

                +---------------------+
                |    API Gateway /     |
                |    Ingress Proxy     |
                +----------+----------+
                           |
          +----------------+----------------+
          |                                 |
  +-------v-------+                 +-------v-------+
  | Service A     |                 | Service B     |
  | +-----------+ |                 | +-----------+ |
  | | Sidecar   | |                 | | Sidecar   | |
  | | Proxy     | |                 | | Proxy     | |
  | +-----------+ |                 | +-----------+ |
  +---------------+                 +---------------+
          |                                 |
          +----------------+----------------+
                           |
                +----------v----------+
                |  Service Registry   |
                +---------------------+
                           |
                +----------v----------+
                | Distributed Tracing |
                +---------------------+

Components

Service Registry

Consul / Eureka / etcd

Keeps track of available service instances and their network locations for discovery

API Gateway / Ingress Proxy

NGINX / Envoy / Kong

Entry point for external requests, routes to appropriate services, handles load balancing

Sidecar Proxy

Envoy sidecar

Runs alongside each service instance to handle service-to-service communication, retries, circuit breaking, and security

Secure Communication

mTLS (mutual TLS)

Encrypts and authenticates communication between services

Circuit Breaker and Retry

Resilience4j / Istio features

Prevents cascading failures by stopping calls to failing services and retrying when appropriate

Distributed Tracing

Jaeger / Zipkin

Tracks requests across services to monitor latency and troubleshoot issues

Request Flow

1. Client sends request to API Gateway

2. API Gateway routes request to Service A via load balancing

3. Service A sidecar proxy discovers Service B location from Service Registry

4. Service A sidecar establishes secure mTLS connection to Service B sidecar

5. Service A calls Service B through sidecar proxy with retries and circuit breaker enabled

6. Distributed tracing headers propagate through calls for monitoring

7. Service B processes request and responds back through sidecar proxies

8. Response flows back to client via API Gateway

Database Schema

Not applicable as this design focuses on service networking and communication, not data storage.

Scaling Discussion

Bottlenecks

Service Registry becoming a single point of failure or bottleneck

API Gateway overload with high incoming traffic

Sidecar proxies adding latency or resource overhead

Network bandwidth limits between services

Tracing system storage and query performance at scale

Solutions

Deploy Service Registry in a highly available cluster with leader election

Use multiple API Gateway instances behind a load balancer

Optimize sidecar resource usage and use lightweight proxies

Use efficient protocols like gRPC and compress payloads

Implement sampling in tracing and use scalable storage backends

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing components and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain importance of dynamic service discovery in microservices

Discuss pros and cons of client-side vs server-side load balancing

Highlight security with mutual TLS between services

Describe how circuit breakers improve system resilience

Show understanding of distributed tracing for observability

Address scaling challenges and practical solutions

Practice

1. What is the main purpose of service discovery in a microservices architecture?

easy

A. To manage database transactions

B. To store user data securely

C. To help services find and communicate with each other dynamically

D. To handle user authentication

Solution

1. Step 1: Understand service discovery role

   Service discovery allows services to locate each other without hardcoding addresses.

2. Step 2: Match purpose with options

   Only To help services find and communicate with each other dynamically describes dynamic service location, others relate to different concerns.

3. Final Answer:

   To help services find and communicate with each other dynamically -> Option C

4. Quick Check:

   Service discovery = dynamic service location [OK]

Hint: Service discovery = finding services automatically [OK]

Common Mistakes:

• Confusing service discovery with authentication
• Thinking it manages databases
• Assuming it stores user data

2. Which of the following is the correct way to specify a REST API call in microservices networking?

easy

A. FETCH /users/api HTTP/1.0

B. POST /api/v1/users HTTP/1.1

C. CONNECT users /api/v1 HTTP/2

D. SEND /api/users HTTP/1.1

Solution

1. Step 1: Identify standard HTTP methods and syntax

   REST APIs use standard HTTP methods like GET, POST, PUT, DELETE with URI paths and HTTP version.

2. Step 2: Match correct syntax

   POST /api/v1/users HTTP/1.1 uses POST method, valid URI, and HTTP/1.1 version correctly; others use invalid methods or syntax.

3. Final Answer:

   POST /api/v1/users HTTP/1.1 -> Option B

4. Quick Check:

   REST API call = HTTP method + URI + version [OK]

Hint: REST calls use standard HTTP verbs and URIs [OK]

Common Mistakes:

• Using non-standard HTTP methods
• Incorrect URI format
• Wrong HTTP version syntax

3. Given the following microservice call sequence:

Service A calls Service B via HTTP.
Service B calls Service C via gRPC.
Service C responds with data.

What is the correct order of communication protocols used in this flow?

medium

A. Only gRPC

B. gRPC then HTTP

C. Only HTTP

D. HTTP then gRPC

Solution

1. Step 1: Trace the call sequence

   Service A calls B using HTTP first, then B calls C using gRPC.

2. Step 2: Identify protocol order

   The communication starts with HTTP and then switches to gRPC.

3. Final Answer:

   HTTP then gRPC -> Option D

4. Quick Check:

   Call sequence protocols = HTTP then gRPC [OK]

Hint: Follow call chain to list protocols in order [OK]

Common Mistakes:

• Mixing protocol order
• Assuming only one protocol is used
• Ignoring protocol differences

4. A microservice is failing to connect to another service using its hardcoded IP address. What is the most likely cause and fix?

medium

A. IP address changed; use service discovery instead of hardcoding

B. Service is down; restart the service

C. Network cable unplugged; check physical connections

D. Firewall blocking traffic; disable firewall

Solution

1. Step 1: Identify problem with hardcoded IP

   Hardcoded IPs break when services move or scale, causing connection failures.

2. Step 2: Recommend dynamic service discovery

   Using service discovery allows services to find current addresses dynamically, fixing the issue.

3. Final Answer:

   IP address changed; use service discovery instead of hardcoding -> Option A

4. Quick Check:

   Hardcoded IP failure = use service discovery [OK]

Hint: Avoid hardcoded IPs; use service discovery [OK]

Common Mistakes:

• Restarting services without checking addresses
• Ignoring dynamic environment changes
• Disabling firewall without cause

5. You are designing a microservices system where services must communicate securely and efficiently. Which combination of networking components is best to ensure service discovery, secure communication, and load balancing?

hard

A. Service registry for discovery, TLS for security, and API gateway for load balancing

B. Static IPs for discovery, HTTP for communication, and DNS for load balancing

C. Manual config files for discovery, plain TCP sockets, and round-robin DNS

D. No discovery needed, use UDP for speed, and client-side load balancing

Solution

1. Step 1: Identify components for service discovery

   A service registry dynamically tracks services, enabling discovery.

2. Step 2: Choose secure communication and load balancing

   TLS encrypts data for security; API gateway can handle load balancing efficiently.

3. Step 3: Evaluate other options

   Static IPs and manual configs lack flexibility; plain TCP and UDP lack security; DNS load balancing is limited.

4. Final Answer:

   Service registry for discovery, TLS for security, and API gateway for load balancing -> Option A

5. Quick Check:

   Discovery + TLS + API gateway = secure scalable system [OK]

Hint: Combine registry, TLS, and gateway for best networking [OK]

Common Mistakes:

• Using static IPs instead of dynamic discovery
• Ignoring encryption needs
• Relying solely on DNS for load balancing