Container networking in Microservices - System Design Exercise

Design: Container Networking System

Design the networking layer for containerized microservices including intra-host and inter-host communication, service discovery, and network isolation. Out of scope are container orchestration scheduling and storage networking.

Functional Requirements

FR1: Enable communication between containers within the same host

FR2: Enable communication between containers across different hosts

FR3: Support service discovery for containers to find each other dynamically

FR4: Provide network isolation and security between different container groups

FR5: Allow containers to expose ports to external clients

FR6: Support scalability to thousands of containers

FR7: Ensure low latency and high throughput for container communication

Non-Functional Requirements

NFR1: Handle up to 10,000 containers across multiple hosts

NFR2: Network latency p99 should be under 10ms for container-to-container communication

NFR3: Availability of networking should be 99.9%

NFR4: Support dynamic container lifecycle (start, stop, move) without manual network reconfiguration

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

❓ Question 7

Key Components

Container network interface (CNI) plugins

Overlay network technologies (e.g., VXLAN, GRE)

Service discovery components (e.g., DNS, etcd, Consul)

Network proxies and load balancers

Network policy enforcement modules

IP address management system

Host network stack and virtual interfaces

Design Patterns

Overlay networking for cross-host container communication

Service mesh for advanced routing and security

Sidecar proxy pattern for network traffic control

Network namespaces for isolation

Load balancing and ingress routing

Dynamic IP allocation and DHCP for containers

Reference Architecture

 +----------------+       +----------------+       +----------------+
 |   Container    |       |   Container    |       |   Container    |
 |    Host A      |       |    Host B      |       |    Host C      |
 | +------------+ |       | +------------+ |       | +------------+ |
 | | Container  | |       | | Container  | |       | | Container  | |
 | | Network    | |       | | Network    | |       | | Network    | |
 | | Namespace  | |       | | Namespace  | |       | | Namespace  | |
 | +------------+ |       | +------------+ |       | +------------+ |
 |      |         |       |      |         |       |      |         |
 | +------------+ |       | +------------+ |       | +------------+ |
 | | CNI Plugin | |       | | CNI Plugin | |       | | CNI Plugin | |
 | +------------+ |       | +------------+ |       | +------------+ |
 +-------|--------+       +-------|--------+       +-------|--------+
         |                        |                        |
         | Overlay Network (VXLAN/GRE) connecting hosts across data center
         |                        |                        |
 +-------v--------------------------------------------------------v-------+
 |                          Physical Network / Cloud Network               |
 +-------------------------------------------------------------------------+

 +----------------+       +----------------+       +----------------+
 | Service        |       | Network Policy |       | Service        |
 | Discovery (DNS)|       | Controller     |       | Mesh / Proxy   |
 +----------------+       +----------------+       +----------------+

Components

Container Network Interface (CNI) Plugin

Calico, Flannel, Weave

Manages container network namespaces, assigns IPs, and sets up virtual interfaces for containers on each host.

Overlay Network

VXLAN, GRE

Creates a virtual network over the physical network to enable container communication across hosts.

Service Discovery

CoreDNS, etcd

Allows containers to find each other dynamically by resolving service names to IP addresses.

Network Policy Controller

Kubernetes Network Policies, Calico Policies

Enforces security rules and network isolation between containers and services.

Load Balancer / Ingress Controller

Envoy, NGINX, HAProxy

Routes external traffic to container services and balances load among container instances.

IP Address Management (IPAM)

Built-in CNI IPAM or external IPAM tools

Allocates and tracks IP addresses for containers to avoid conflicts.

Request Flow

1. 1. Container starts on a host; CNI plugin assigns an IP and sets up network namespace.

2. 2. Container registers its service and IP with the service discovery system.

3. 3. When a container wants to communicate with another, it queries service discovery for the target IP.

4. 4. The container sends packets through its virtual interface; packets are encapsulated by the overlay network if the destination is on a different host.

5. 5. Overlay network routes packets across the physical network to the destination host.

6. 6. Network policy controller checks if communication is allowed based on defined rules.

7. 7. Packets are delivered to the destination container's network namespace.

8. 8. For external access, traffic hits the load balancer or ingress controller, which routes it to the appropriate container.

Database Schema

Entities: - Container: id (PK), host_id (FK), ip_address, network_namespace, status - Host: id (PK), hostname, physical_ip - Service: id (PK), name, selector_labels - ServiceInstance: id (PK), service_id (FK), container_id (FK), ip_address - NetworkPolicy: id (PK), name, rules (JSON), applied_to (labels) Relationships: - Host 1:N Container - Service 1:N ServiceInstance - NetworkPolicy applies to Containers via labels

Scaling Discussion

Bottlenecks

Overlay network encapsulation overhead causing latency increase at high scale

Service discovery becoming a bottleneck with thousands of service instances

IP address exhaustion in flat IP space

Network policy enforcement performance degradation with many rules

Load balancer capacity limits under heavy external traffic

Solutions

Use efficient overlay protocols and hardware offloading to reduce encapsulation overhead

Implement hierarchical or sharded service discovery to distribute load

Use multiple IP address pools or IPv6 to expand address space

Optimize network policy rules and use scalable enforcement engines

Deploy multiple load balancers with DNS-based load distribution and autoscaling

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, and 5 minutes summarizing.

Explain how container networking differs from traditional networking

Discuss the role of CNI plugins and overlay networks

Highlight service discovery integration and dynamic IP management

Emphasize network isolation and security via policies

Address scalability challenges and realistic solutions

Mention trade-offs between complexity and performance