Istio overview in Microservices - System Design Exercise

Design: Istio Service Mesh

Design the architecture of Istio as a service mesh for microservices communication management. Out of scope: detailed implementation of each microservice.

Functional Requirements

FR1: Manage communication between microservices securely and reliably

FR2: Provide traffic routing and load balancing

FR3: Enable observability with monitoring and tracing

FR4: Support policy enforcement and access control

FR5: Handle failure recovery and retries automatically

Non-Functional Requirements

NFR1: Must support thousands of microservices instances

NFR2: Latency added by service mesh should be less than 5ms per request

NFR3: Availability target of 99.9% uptime for service communication

NFR4: Should integrate with Kubernetes environments

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Envoy sidecar proxies

Istio control plane components (Pilot, Mixer, Citadel)

Service registry and discovery

Telemetry and tracing systems

Policy enforcement modules

Design Patterns

Sidecar proxy pattern

Control plane and data plane separation

Circuit breaker and retry patterns

Distributed tracing and metrics collection

Policy-based access control

Reference Architecture

User Client
   |
   v
[Microservice A] <--> [Envoy Sidecar Proxy] <--> Istio Data Plane <--> [Envoy Sidecar Proxy] <--> [Microservice B]
   ^                                         |
   |                                         v
Istio Control Plane (Pilot, Mixer, Citadel)
   |
   v
Telemetry & Policy Systems

Components

Envoy Sidecar Proxy

Envoy Proxy

Intercepts all inbound and outbound traffic for microservices to manage routing, retries, and security

Pilot

Istio Control Plane Component

Distributes traffic management rules and service discovery information to Envoy proxies

Mixer

Istio Control Plane Component

Enforces access control and usage policies, collects telemetry data

Citadel

Istio Control Plane Component

Manages service identity and issues certificates for mutual TLS authentication

Telemetry & Tracing Systems

Prometheus, Jaeger

Collects metrics and traces for observability and debugging

Request Flow

1. Client sends request to Microservice A

2. Envoy sidecar proxy intercepts request and applies routing rules from Pilot

3. Proxy enforces security policies from Citadel (e.g., mutual TLS)

4. Request forwarded through Istio data plane to destination Envoy proxy

5. Destination proxy applies policies and forwards request to Microservice B

6. Mixer collects telemetry data and enforces policies during request

7. Telemetry data sent to monitoring systems for analysis

Database Schema

Not applicable as Istio is a service mesh managing communication and policies rather than storing application data.

Scaling Discussion

Bottlenecks

Envoy sidecar proxy CPU and memory usage under high traffic

Control plane components becoming a single point of failure

Telemetry data volume overwhelming storage and processing

Latency added by multiple proxy hops

Solutions

Scale Envoy proxies horizontally by distributing microservice instances

Deploy control plane components in highly available clusters

Use sampling and aggregation to reduce telemetry data volume

Optimize proxy configurations and use local caching to reduce latency

Interview Tips

Time: Spend 10 minutes explaining Istio components and their roles, 15 minutes on request flow and security, 10 minutes on scaling challenges and solutions, and 10 minutes on real-world use cases.

Clear separation of control plane and data plane

Role of Envoy sidecar proxies in managing traffic

How Istio provides security with mutual TLS

Observability features with telemetry and tracing

Scalability considerations and fault tolerance