Data consistency challenges in Microservices - System Design Guide

Problem Statement

When multiple microservices update related data independently, inconsistencies arise because changes in one service may not immediately reflect in others. This leads to stale or conflicting data views, causing errors like double spending, incorrect inventory counts, or mismatched user profiles.

Solution

To handle this, microservices use patterns like eventual consistency and distributed transactions. They coordinate updates through asynchronous messaging or orchestration to ensure all services eventually agree on the data state, even if temporary inconsistencies occur.

Architecture

Service A

(Order Mgmt)

→Message Bus

↓

Database A

This diagram shows two microservices updating their own databases and communicating asynchronously via a message bus to keep data eventually consistent.

Trade-offs

✓ Pros

→

Allows services to operate independently without blocking each other.

→

Improves system availability by avoiding distributed locks.

→

Scales well as each service manages its own data.

✗ Cons

→

Temporary data inconsistencies can confuse users or cause errors.

→

Complexity increases due to asynchronous coordination and failure handling.

→

Debugging and testing become harder because of eventual consistency delays.

Use when microservices own separate data stores and need to update related data without sacrificing availability, especially at scale above thousands of requests per second.

Avoid when strong immediate consistency is required, such as financial transactions needing atomic commits, or when system scale is small and simpler synchronous updates suffice.

Real World Examples

Amazon

Uses eventual consistency between order and inventory services to handle high volume orders without locking inventory databases.

Uber

Coordinates trip and payment microservices asynchronously to keep user and driver data consistent despite network delays.

Netflix

Manages user profiles and viewing history across microservices with eventual consistency to maintain high availability.

Code Example

This example shows how replacing direct synchronous calls with asynchronous event publishing decouples services and enables eventual consistency in microservices.

Microservices

### Before: Synchronous update causing tight coupling and blocking
class OrderService:
    def create_order(self, order):
        self.db.save(order)
        inventory_service.update_stock(order.item_id, -order.quantity)  # direct call


### After: Asynchronous event-driven update for eventual consistency
class OrderService:
    def create_order(self, order):
        self.db.save(order)
        event = {'type': 'OrderCreated', 'item_id': order.item_id, 'quantity': order.quantity}
        message_bus.publish(event)

class InventoryService:
    def on_order_created(self, event):
        self.db.update_stock(event['item_id'], -event['quantity'])


# Explanation:
# The before code tightly couples OrderService to InventoryService with synchronous calls,
# risking delays and failures propagating.
# The after code decouples them by publishing an event asynchronously,
# allowing InventoryService to update stock independently, achieving eventual consistency.

Alternatives

Distributed Transactions (Two-Phase Commit)

Coordinates all services to commit or rollback changes atomically using a coordinator.

Use when: Use when strict strong consistency is mandatory and system scale is moderate.

Command Query Responsibility Segregation (CQRS)

Separates read and write models to optimize consistency and performance differently.

Use when: Use when read and write workloads differ significantly and eventual consistency is acceptable.

Summary

Microservices face data consistency challenges because each service manages its own data independently.

Eventual consistency uses asynchronous communication to keep data aligned over time without blocking services.

Choosing the right consistency approach depends on system scale, availability needs, and business requirements.