Parallel running in Microservices - System Design Exercise

Design: Parallel Running Deployment System

Design focuses on the deployment and routing architecture for parallel running of microservices. It excludes detailed implementation of microservice business logic and database schema beyond what supports parallel running.

Functional Requirements

FR1: Support running two versions of a microservice simultaneously in production.

FR2: Route a configurable percentage of user requests to the new version while the rest go to the old version.

FR3: Ensure data consistency between versions during parallel operation.

FR4: Allow quick rollback to the old version if issues occur in the new version.

FR5: Monitor performance and errors separately for each version.

FR6: Minimize downtime during deployment and switching.

Non-Functional Requirements

NFR1: Handle up to 10,000 concurrent users with low latency (p99 < 200ms).

NFR2: Availability target of 99.9% uptime (less than 8.77 hours downtime per year).

NFR3: Data consistency must be eventual but with mechanisms to detect divergence.

NFR4: Deployment changes should not cause user-visible errors or downtime.

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

API Gateway or Load Balancer with traffic splitting

Service Registry and Discovery

Centralized Logging and Monitoring

Database with version-aware schema or synchronization

Deployment Automation (CI/CD pipeline)

Feature Flags or Configuration Management

Design Patterns

Blue-Green Deployment

Canary Releases

Feature Toggles

Eventual Consistency

Circuit Breaker for fault tolerance

Reference Architecture

                +-------------------+
                |   API Gateway     |
                | (Traffic Splitter)|
                +---------+---------+
                          |
          +---------------+---------------+
          |                               |
+---------v---------+           +---------v---------+
| Old Version       |           | New Version       |
| Microservice      |           | Microservice      |
+---------+---------+           +---------+---------+
          |                               |
          +---------------+---------------+
                          |
                +---------v---------+
                | Shared Database    |
                | (Version-aware or  |
                |  synchronized)     |
                +-------------------+

Additional components:
- Monitoring & Logging system collects metrics from both versions.
- Deployment Automation manages rollout and rollback.
- Configuration Service controls traffic split ratios.

Components

API Gateway

Kong, NGINX, or AWS API Gateway

Routes incoming requests to old or new microservice versions based on configured traffic split.

Old Version Microservice

Docker container or Kubernetes pod

Handles requests with stable, currently deployed version.

New Version Microservice

Docker container or Kubernetes pod

Handles requests for new version during parallel running.

Shared Database

PostgreSQL or MongoDB with version-aware schema or synchronization

Stores data accessible by both versions, ensuring consistency.

Monitoring & Logging

Prometheus, Grafana, ELK Stack

Collects and visualizes metrics and logs separately for each version.

Deployment Automation

Jenkins, GitHub Actions, ArgoCD

Automates deployment, traffic shifting, and rollback.

Configuration Service

Consul, etcd, or custom config server

Manages traffic split ratios and feature flags.

Request Flow

1. 1. User sends request to API Gateway.

2. 2. API Gateway checks traffic split configuration.

3. 3. Request is routed to either old or new microservice version accordingly.

4. 4. Microservice processes request and reads/writes data to shared database.

5. 5. Both versions log metrics and errors to Monitoring & Logging system.

6. 6. Deployment Automation adjusts traffic split gradually to new version.

7. 7. If issues detected, rollback triggers traffic to old version only.

8. 8. After stable operation, old version is decommissioned.

Database Schema

Entities are designed to support both versions, either by: - Using backward-compatible schema changes, - Or maintaining version tags on records, - Or synchronizing data asynchronously if schema differs. Relationships remain consistent to support both versions' data needs.

Scaling Discussion

Bottlenecks

API Gateway becomes a bottleneck if not horizontally scalable.

Database contention due to simultaneous writes from two versions.

Monitoring system overload with doubled metrics volume.

Deployment automation complexity increases with multiple versions.

Data inconsistency risk if schema changes are incompatible.

Solutions

Use horizontally scalable API Gateway clusters with load balancing.

Implement database sharding or use multi-master replication for write scaling.

Aggregate and sample metrics to reduce monitoring load.

Automate deployment with robust rollback and health checks.

Use feature flags and backward-compatible schema migrations to minimize inconsistency.

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain how traffic splitting enables safe parallel running.

Discuss data consistency challenges and solutions.

Highlight monitoring and rollback importance.

Describe how deployment automation reduces human error.

Mention scalability considerations and how to address bottlenecks.