Eventual consistency in Microservices - System Design Guide

Problem Statement

When multiple services update shared data independently, immediate synchronization can fail, causing temporary data conflicts and stale reads. This leads to user confusion and errors if the system expects all parts to have the exact same data instantly.

Solution

Eventual consistency allows services to update data independently and asynchronously, accepting temporary differences. Over time, updates propagate through the system until all services converge to the same data state, ensuring consistency without blocking operations.

Architecture

┌─────────────┐       ┌─────────────┐       ┌─────────────┐
│ Service A   │──────▶│ Event Bus   │──────▶│ Service B   │
└─────────────┘       └─────────────┘       └─────────────┘
       │                                         ▲
       │                                         │
       └─────────────────────────────────────────┘


Legend:
- Service A publishes updates to Event Bus.
- Service B consumes events and updates its data asynchronously.
- Data converges over time.

This diagram shows how services communicate updates via an event bus asynchronously, allowing data to become consistent eventually.

Trade-offs

✓ Pros

→

Improves system availability by avoiding synchronous locks or waits.

→

Enables high scalability as services operate independently.

→

Reduces latency for write operations since they don't wait for global consensus.

✗ Cons

→

Temporary data inconsistencies can confuse users or cause errors.

→

Requires complex conflict resolution and reconciliation logic.

→

Harder to reason about system state at any given moment.

Use when your system has high write throughput, distributed services, and can tolerate short periods of inconsistency, such as social media feeds or shopping carts.

Avoid when strong consistency is critical, like financial transactions or inventory counts where stale data can cause serious errors.

Real World Examples

Amazon

Uses eventual consistency in its DynamoDB to allow distributed data replication with high availability and partition tolerance.

Netflix

Applies eventual consistency in its microservices to handle user preferences and viewing history updates asynchronously for better scalability.

LinkedIn

Employs eventual consistency in its feed system to propagate updates across distributed services without blocking user interactions.

Code Example

This code shows how to move from blocking synchronous updates to asynchronous event-driven updates. The InventoryService updates its data and publishes an event without waiting for others. The NotificationService listens and updates itself later, allowing data to converge over time.

Microservices

### Before: Synchronous update causing blocking and tight coupling
class InventoryService:
    def update_stock(self, product_id, quantity):
        # Directly update database and block until done
        database.update(product_id, quantity)
        notify_other_services(product_id, quantity)


### After: Eventual consistency with asynchronous event publishing
import asyncio

class InventoryService:
    async def update_stock(self, product_id, quantity):
        # Update local database immediately
        database.update(product_id, quantity)
        # Publish event asynchronously
        await event_bus.publish('stock_updated', {'product_id': product_id, 'quantity': quantity})

class NotificationService:
    async def handle_stock_update(self, event):
        # Consume event and update own state asynchronously
        await database.update(event['product_id'], event['quantity'])

# Explanation:
# The before code blocks until all updates and notifications complete, causing delays.
# The after code updates local data immediately and publishes events asynchronously,
# allowing other services to update later, achieving eventual consistency.

Alternatives

Strong consistency

Requires all nodes to agree on data updates before confirming success, blocking operations until consensus.

Use when: When correctness and immediate data accuracy are critical, such as banking or inventory management.

Read-after-write consistency

Guarantees that a read immediately following a write returns the updated data, but may not scale well in distributed systems.

Use when: When users expect to see their own updates instantly but global consistency is less critical.

Summary

Eventual consistency allows distributed services to update data independently and asynchronously.

It improves availability and scalability by avoiding blocking operations.

Temporary inconsistencies require conflict resolution and are unsuitable for critical data.