Parallel running in Microservices - System Design Guide

Problem Statement

When deploying a new version of a microservice, switching directly can cause system downtime or unexpected failures that impact users. Without a safe way to compare old and new versions, bugs or performance issues may go unnoticed until after full rollout.

Solution

Parallel running deploys the new microservice version alongside the old one, routing a portion of live traffic to both simultaneously. This allows real-time comparison of outputs and performance under production load, enabling safe validation before full cutover.

Architecture

Client/API

→Load Balancer

↓

Old Version

Microservice

This diagram shows a load balancer routing client requests to both old and new microservice versions in parallel for comparison.

Trade-offs

✓ Pros

→

Enables safe validation of new microservice versions under real production load.

→

Reduces risk of downtime by allowing rollback to old version instantly.

→

Helps detect subtle bugs or performance regressions before full deployment.

✗ Cons

→

Requires double infrastructure resources during parallel run period.

→

Increases complexity of traffic routing and result comparison.

→

May introduce data consistency challenges if both versions write to shared storage.

Use when deploying critical microservices where downtime or bugs have high impact, especially at scale above thousands of requests per second.

Avoid when system load is low (under 1000 req/sec) or infrastructure cost constraints prevent running duplicate services.

Real World Examples

Netflix

Netflix uses parallel running to deploy new streaming service versions alongside old ones, ensuring smooth user experience without interruptions.

Uber

Uber runs new dispatch microservices in parallel with existing ones to validate routing logic under live traffic before full rollout.

Amazon

Amazon deploys new versions of order processing microservices in parallel to detect errors early and avoid order failures.

Code Example

The before code uses only the old microservice version, risking downtime on switch. The after code runs both old and new versions in parallel, compares their outputs, and returns the old version's response until the new one is validated.

Microservices

### Before: Direct switch without parallel running
class Microservice:
    def process(self, request):
        # Old version logic
        return "old response"

service = Microservice()
request = {}  # Added to define 'request'
response = service.process(request)


### After: Parallel running with traffic split and comparison
class OldMicroservice:
    def process(self, request):
        return "old response"

class NewMicroservice:
    def process(self, request):
        return "new response"

class ParallelRunner:
    def __init__(self):
        self.old_service = OldMicroservice()
        self.new_service = NewMicroservice()

    def process(self, request):
        old_resp = self.old_service.process(request)
        new_resp = self.new_service.process(request)
        # Compare responses for validation
        if old_resp != new_resp:
            print("Warning: Responses differ")
        # Return old response to client until new version is fully validated
        return old_resp

runner = ParallelRunner()
request = {}  # Added to define 'request'
response = runner.process(request)

Alternatives

Blue-Green Deployment

Blue-green switches all traffic instantly between two environments rather than splitting traffic simultaneously.

Use when: Choose blue-green when instant cutover with minimal parallel resource usage is preferred and rollback speed is critical.

Canary Deployment

Canary gradually shifts a small percentage of traffic to new version instead of running both fully in parallel.

Use when: Choose canary when incremental exposure to new version is needed to limit blast radius of failures.

Summary

Parallel running deploys new and old microservice versions simultaneously to compare outputs under real traffic.

It reduces downtime risk by validating new versions before full cutover and enables quick rollback.

This pattern is best for critical services at scale but requires extra resources and complexity temporarily.