Outbox pattern for reliable events in Microservices - System Design Guide

Problem Statement

When a microservice updates its database and publishes an event to notify other services, doing both in separate steps can cause inconsistencies. If the service crashes after updating the database but before sending the event, other services never learn about the change, leading to data mismatches and lost messages.

Solution

The outbox pattern solves this by writing the event data into a special 'outbox' table within the same database transaction as the business update. A separate process then reliably reads the outbox table and publishes the events to the message broker, ensuring no event is lost and the database and event stream stay consistent.

Architecture

Client Request

→Microservice

↓

Outbox Processor

Outbox Processor

↓

Message Broker

Message Broker

This diagram shows the client request triggering a microservice that updates its database and writes an event to the outbox table in one transaction. The outbox processor then reads events from the outbox table and publishes them to the message broker.

Trade-offs

✓ Pros

→

Ensures atomicity between database updates and event creation, preventing lost events.

→

Decouples event publishing from business logic, improving reliability and scalability.

→

Simplifies recovery by replaying events from the outbox after failures.

✗ Cons

→

Requires additional storage and management of the outbox table.

→

Adds complexity with a separate outbox processor component.

→

May introduce slight latency between database commit and event publication.

Use when microservices need strong consistency between database state and events, especially at scales above hundreds of requests per second where lost events cause critical issues.

Avoid when the system has very low event volume (under 100 req/sec) or eventual consistency is acceptable, as the added complexity may not justify the benefits.

Real World Examples

Uber

Uber uses the outbox pattern to ensure ride state changes are reliably published as events, preventing mismatches between ride data and notifications.

Shopify

Shopify applies the outbox pattern to guarantee order updates are atomically saved and corresponding events are published for inventory and billing services.

LinkedIn

LinkedIn uses the outbox pattern to maintain consistency between user profile updates and event streams consumed by search and recommendation systems.

Code Example

The before code updates the order and publishes an event separately, risking lost events if a crash occurs between steps. The after code writes the order update and event to the outbox table in one transaction, ensuring atomicity. A separate processor reliably publishes events from the outbox.

Microservices

### Before (without outbox pattern):
# Update DB and publish event separately

def update_order(order_id, new_status):
    db.execute("UPDATE orders SET status = %s WHERE id = %s", (new_status, order_id))
    # If crash happens here, event is lost
    event_bus.publish({"order_id": order_id, "status": new_status})


### After (with outbox pattern):
# Write update and event in one DB transaction

def update_order(order_id, new_status):
    with db.transaction() as txn:
        txn.execute("UPDATE orders SET status = %s WHERE id = %s", (new_status, order_id))
        txn.execute("INSERT INTO outbox (event_type, payload) VALUES (%s, %s)",
                    ("OrderStatusChanged", json.dumps({"order_id": order_id, "status": new_status})))

# Separate process reads outbox and publishes events

def outbox_processor():
    events = db.query("SELECT id, event_type, payload FROM outbox WHERE published = FALSE")
    for event in events:
        event_bus.publish(json.loads(event.payload))
        db.execute("UPDATE outbox SET published = TRUE WHERE id = %s", (event.id,))

Alternatives

Two-phase commit (2PC)

2PC coordinates distributed transactions across services to ensure atomic commits, unlike the outbox pattern which relies on local DB transactions plus asynchronous event publishing.

Use when: Choose 2PC when strict distributed transaction atomicity is required and system complexity can be tolerated.

Event sourcing

Event sourcing stores all state changes as events first, making events the source of truth, whereas outbox pattern stores state in DB and events separately.

Use when: Choose event sourcing when auditability and replayability of all state changes are primary requirements.

Change Data Capture (CDC)

CDC captures database changes from logs and publishes events externally, while outbox pattern explicitly writes events in the same transaction as business data.

Use when: Choose CDC when you want to avoid modifying application code and can rely on database log streaming.

Summary

The outbox pattern prevents lost events by writing events and database updates in a single transaction.

It uses a separate process to read and publish events from the outbox table reliably.

This pattern improves consistency between microservices but adds some complexity and latency.