Eventual consistency handling in Microservices - System Design Guide

Problem Statement

When multiple microservices update shared data independently, immediate consistency is hard to guarantee. This causes temporary data mismatches, stale reads, or conflicting updates that confuse users and break workflows.

Solution

Eventual consistency handling allows services to update data asynchronously and propagate changes over time. Services communicate changes via events or messages, ensuring all parts eventually see the same data without blocking operations or requiring tight coordination.

Architecture

Service A

→Event Bus

↓

Database A

This diagram shows two microservices updating their own databases and communicating changes asynchronously through an event bus to achieve eventual consistency.

Trade-offs

✓ Pros

→

Improves system availability by avoiding synchronous blocking calls between services.

→

Allows services to operate independently, improving scalability and fault tolerance.

→

Reduces coupling between services, enabling easier maintenance and deployment.

✗ Cons

→

Data may be temporarily inconsistent, leading to stale reads or conflicts.

→

Requires complex conflict resolution and retry mechanisms.

→

Debugging and tracing data flow becomes harder due to asynchronous nature.

Use when your system has multiple distributed services that update shared data and can tolerate short periods of inconsistency, typically at scales above thousands of requests per second.

Avoid when your application requires strong consistency guarantees, such as financial transactions or critical inventory updates where stale data causes unacceptable errors.

Real World Examples

Amazon

Amazon uses eventual consistency in its order processing microservices to allow independent updates of inventory and payment services without blocking user experience.

Uber

Uber applies eventual consistency to synchronize ride status updates across driver and rider services, allowing high availability despite network delays.

Netflix

Netflix uses eventual consistency to update user preferences and viewing history asynchronously across multiple services to maintain responsiveness.

Code Example

The before code shows a synchronous call from OrderService to InventoryService, causing blocking and tight coupling. The after code publishes an event asynchronously, allowing InventoryService to update stock independently, enabling eventual consistency.

Microservices

### Before: Synchronous update causing blocking and tight coupling
class OrderService:
    def create_order(self, order_data):
        # Update order database
        self.db.save(order_data)
        # Synchronously call inventory service
        success = self.inventory_service.reserve_stock(order_data['items'])
        if not success:
            raise Exception('Stock reservation failed')


### After: Asynchronous event-driven update for eventual consistency
class OrderService:
    def create_order(self, order_data):
        # Update order database
        self.db.save(order_data)
        # Publish event to event bus
        event = {'type': 'OrderCreated', 'data': order_data}
        self.event_bus.publish(event)

class InventoryService:
    def on_order_created(self, event):
        items = event['data']['items']
        # Reserve stock asynchronously
        self.db.reserve_stock(items)

# Event bus delivers events asynchronously, decoupling services

Alternatives

Strong consistency

Requires synchronous coordination and locking to ensure all services see the same data immediately.

Use when: Choose when your system cannot tolerate any stale or conflicting data, such as banking or inventory control.

Two-phase commit

A distributed transaction protocol that locks resources and commits changes atomically across services.

Use when: Choose when you need atomic updates across multiple services but can tolerate higher latency and complexity.

Summary

Eventual consistency allows distributed services to update data asynchronously, avoiding blocking and tight coupling.

It improves availability and scalability but requires handling temporary data inconsistencies and conflicts.

Use it when your system can tolerate short delays in data synchronization and needs to scale across multiple services.