Services and networking in Microservices - System Design Guide

Problem Statement

When multiple microservices need to communicate, relying on direct, hardcoded connections causes tight coupling and brittle systems. Without proper networking and service management, requests can fail silently, cause delays, or overload services, leading to poor reliability and scalability.

Solution

Services communicate through well-defined network protocols using service discovery and load balancing. Requests are routed dynamically to healthy instances, and communication can be synchronous or asynchronous. This setup decouples services, improves fault tolerance, and enables scaling by adding or removing service instances transparently.

Architecture

Client App

→API Gateway

↓

Service B

This diagram shows a client sending requests to an API Gateway, which routes requests to multiple microservices (A, B, C). The gateway manages networking and load balancing between services.

Trade-offs

✓ Pros

→

Decouples services allowing independent deployment and scaling.

→

Improves fault tolerance by routing around failed instances.

→

Enables dynamic discovery of services without hardcoded addresses.

→

Supports multiple communication patterns (sync, async).

✗ Cons

→

Adds network latency compared to in-process calls.

→

Increases system complexity with service discovery and routing layers.

→

Requires careful security and authentication management across services.

Use when building distributed microservices architecture with multiple services needing reliable communication and scalability beyond a few instances.

Avoid if your system is a simple monolith or has fewer than 5 services with low traffic, where added networking complexity outweighs benefits.

Real World Examples

Netflix

Uses an API Gateway and service discovery to route millions of streaming requests reliably to thousands of microservice instances.

Uber

Employs service mesh and dynamic routing to connect ride-matching, payments, and notifications services with low latency and high availability.

Amazon

Uses service discovery and load balancing to manage communication between inventory, order, and payment microservices at massive scale.

Code Example

The before code calls a fixed service address, causing tight coupling and no failover. The after code uses a simple load balancer to pick among multiple service instances, enabling better availability and scalability.

Microservices

### Before: Direct service call with hardcoded address
import requests

def get_user_profile(user_id):
    response = requests.get(f'http://service-a:5000/users/{user_id}')
    return response.json()


### After: Using service discovery and load balancing
import requests

SERVICE_A_INSTANCES = ['http://10.0.0.1:5000', 'http://10.0.0.2:5000']

class LoadBalancer:
    def __init__(self, instances):
        self.instances = instances
        self.index = 0

    def get_instance(self):
        instance = self.instances[self.index]
        self.index = (self.index + 1) % len(self.instances)
        return instance

load_balancer = LoadBalancer(SERVICE_A_INSTANCES)

def get_user_profile(user_id):
    base_url = load_balancer.get_instance()
    response = requests.get(f'{base_url}/users/{user_id}')
    return response.json()

Alternatives

Monolithic Architecture

All components run in a single process without network calls between services.

Use when: Choose when your application is small, simple, and does not require independent scaling or deployment.

Serverless Functions

Functions are event-driven and stateless, communicating via events or APIs without persistent service instances.

Use when: Choose when you want to minimize infrastructure management and have highly variable workloads.

Summary

Networking in microservices prevents tight coupling by enabling dynamic service discovery and routing.

It improves reliability by load balancing requests and handling failures gracefully.

Proper networking supports scaling and independent deployment of services.

Practice

1. What is the main purpose of service discovery in a microservices architecture?

easy

A. To manage database transactions

B. To store user data securely

C. To help services find and communicate with each other dynamically

D. To handle user authentication

Solution

1. Step 1: Understand service discovery role

   Service discovery allows services to locate each other without hardcoding addresses.

2. Step 2: Match purpose with options

   Only To help services find and communicate with each other dynamically describes dynamic service location, others relate to different concerns.

3. Final Answer:

   To help services find and communicate with each other dynamically -> Option C

4. Quick Check:

   Service discovery = dynamic service location [OK]

Hint: Service discovery = finding services automatically [OK]

Common Mistakes:

• Confusing service discovery with authentication
• Thinking it manages databases
• Assuming it stores user data

2. Which of the following is the correct way to specify a REST API call in microservices networking?

easy

A. FETCH /users/api HTTP/1.0

B. POST /api/v1/users HTTP/1.1

C. CONNECT users /api/v1 HTTP/2

D. SEND /api/users HTTP/1.1

Solution

1. Step 1: Identify standard HTTP methods and syntax

   REST APIs use standard HTTP methods like GET, POST, PUT, DELETE with URI paths and HTTP version.

2. Step 2: Match correct syntax

   POST /api/v1/users HTTP/1.1 uses POST method, valid URI, and HTTP/1.1 version correctly; others use invalid methods or syntax.

3. Final Answer:

   POST /api/v1/users HTTP/1.1 -> Option B

4. Quick Check:

   REST API call = HTTP method + URI + version [OK]

Hint: REST calls use standard HTTP verbs and URIs [OK]

Common Mistakes:

• Using non-standard HTTP methods
• Incorrect URI format
• Wrong HTTP version syntax

3. Given the following microservice call sequence:

Service A calls Service B via HTTP.
Service B calls Service C via gRPC.
Service C responds with data.

What is the correct order of communication protocols used in this flow?

medium

A. Only gRPC

B. gRPC then HTTP

C. Only HTTP

D. HTTP then gRPC

Solution

1. Step 1: Trace the call sequence

   Service A calls B using HTTP first, then B calls C using gRPC.

2. Step 2: Identify protocol order

   The communication starts with HTTP and then switches to gRPC.

3. Final Answer:

   HTTP then gRPC -> Option D

4. Quick Check:

   Call sequence protocols = HTTP then gRPC [OK]

Hint: Follow call chain to list protocols in order [OK]

Common Mistakes:

• Mixing protocol order
• Assuming only one protocol is used
• Ignoring protocol differences

4. A microservice is failing to connect to another service using its hardcoded IP address. What is the most likely cause and fix?

medium

A. IP address changed; use service discovery instead of hardcoding

B. Service is down; restart the service

C. Network cable unplugged; check physical connections

D. Firewall blocking traffic; disable firewall

Solution

1. Step 1: Identify problem with hardcoded IP

   Hardcoded IPs break when services move or scale, causing connection failures.

2. Step 2: Recommend dynamic service discovery

   Using service discovery allows services to find current addresses dynamically, fixing the issue.

3. Final Answer:

   IP address changed; use service discovery instead of hardcoding -> Option A

4. Quick Check:

   Hardcoded IP failure = use service discovery [OK]

Hint: Avoid hardcoded IPs; use service discovery [OK]

Common Mistakes:

• Restarting services without checking addresses
• Ignoring dynamic environment changes
• Disabling firewall without cause

5. You are designing a microservices system where services must communicate securely and efficiently. Which combination of networking components is best to ensure service discovery, secure communication, and load balancing?

hard

A. Service registry for discovery, TLS for security, and API gateway for load balancing

B. Static IPs for discovery, HTTP for communication, and DNS for load balancing

C. Manual config files for discovery, plain TCP sockets, and round-robin DNS

D. No discovery needed, use UDP for speed, and client-side load balancing

Solution

1. Step 1: Identify components for service discovery

   A service registry dynamically tracks services, enabling discovery.

2. Step 2: Choose secure communication and load balancing

   TLS encrypts data for security; API gateway can handle load balancing efficiently.

3. Step 3: Evaluate other options

   Static IPs and manual configs lack flexibility; plain TCP and UDP lack security; DNS load balancing is limited.

4. Final Answer:

   Service registry for discovery, TLS for security, and API gateway for load balancing -> Option A

5. Quick Check:

   Discovery + TLS + API gateway = secure scalable system [OK]

Hint: Combine registry, TLS, and gateway for best networking [OK]

Common Mistakes:

• Using static IPs instead of dynamic discovery
• Ignoring encryption needs
• Relying solely on DNS for load balancing