Event sourcing pattern in Microservices - System Design Exercise

Design: Event Sourcing System

Design the event sourcing pattern implementation within a microservices architecture including event storage, event publishing, state rebuilding, and querying. Out of scope: UI design, specific business logic details.

Functional Requirements

FR1: Store all changes to application state as a sequence of events.

FR2: Allow rebuilding current state by replaying events.

FR3: Support querying current state efficiently.

FR4: Ensure events are immutable and append-only.

FR5: Support multiple microservices consuming events asynchronously.

FR6: Provide audit trail for all changes.

FR7: Handle concurrent updates safely.

Non-Functional Requirements

NFR1: System must handle 10,000 events per second.

NFR2: Event replay latency should be under 5 seconds for rebuilding state.

NFR3: Availability target of 99.9% uptime.

NFR4: Event storage must be durable and scalable.

NFR5: APIs should respond within 200ms p99 latency.

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

❓ Question 7

Key Components

Event Store (append-only log database)

Command API to accept state changes

Event Publisher / Message Broker

Event Consumers / Microservices

Snapshot Store for faster state rebuilding

Read Model / Query Database

Concurrency control mechanisms

Design Patterns

Event Sourcing

CQRS (Command Query Responsibility Segregation)

Publish-Subscribe messaging

Snapshotting

Idempotent event processing

Reference Architecture

          +-------------------+          
          |   Client / UI     |          
          +---------+---------+          
                    |                    
                    v                    
          +---------+---------+          
          |   Command API     |          
          +---------+---------+          
                    |                    
                    v                    
          +---------+---------+          
          |    Event Store    |<---------+  
          +---------+---------+          |  
                    |                    |  
                    v                    |  
          +---------+---------+          |  
          | Event Publisher   |          |  
          +---------+---------+          |  
                    |                    |  
        +-----------+-----------+        |  
        |                       |        |  
+-------v-------+       +-------v-------+|  
| Read Model DB |       | Microservice 1 ||  
+---------------+       +---------------+|  
                                         |  
          +-------------------------------+  
          |       Snapshot Store           
          +-------------------------------+  

Components

Command API

REST/gRPC microservice

Accepts commands from clients to change state, validates and converts them into events.

Event Store

Append-only log database (e.g., Apache Kafka, EventStoreDB)

Stores all events immutably in order for replay and audit.

Event Publisher

Message broker (e.g., Kafka, RabbitMQ)

Publishes events to subscribers asynchronously.

Microservices (Event Consumers)

Microservices subscribing to event streams

Consume events to update their own state or trigger actions.

Read Model Database

NoSQL or relational DB optimized for queries

Stores current state projections built from events for fast querying.

Snapshot Store

Key-value store or database

Stores periodic snapshots of state to speed up rebuilding from events.

Request Flow

1. Client sends a command to Command API to change state.

2. Command API validates command and creates one or more events.

3. Events are appended to the Event Store in order.

4. Event Publisher reads new events from Event Store and publishes them to message broker.

5. Microservices subscribe to event streams and consume events asynchronously.

6. Microservices update their own state or trigger side effects based on events.

7. Read Model Database is updated by event consumers to reflect current state.

8. Snapshot Store periodically saves snapshots of state to optimize rebuilding.

9. When rebuilding state, system loads latest snapshot and replays subsequent events.

Database Schema

Entities: - Event: {event_id (PK), aggregate_id, event_type, event_data (JSON), timestamp, version} - Snapshot: {snapshot_id (PK), aggregate_id, snapshot_data (JSON), last_event_version, timestamp} - Aggregate: {aggregate_id (PK), current_version, last_snapshot_version} Relationships: - One Aggregate has many Events (1:N) - One Aggregate has many Snapshots (1:N) - Events are immutable and append-only - Snapshots reference the last event version included

Scaling Discussion

Bottlenecks

Event Store write throughput limits at very high event rates.

Event replay latency grows with event history size.

Message broker overload with many subscribers or high event volume.

Read Model DB query performance under heavy load.

Snapshot creation impacting system performance.

Solutions

Partition Event Store by aggregate or topic to increase write throughput.

Use snapshotting to reduce event replay time for rebuilding state.

Scale message broker clusters and use consumer groups for load balancing.

Use read model databases optimized for query patterns and scale horizontally.

Schedule snapshot creation during low traffic periods and use incremental snapshots.

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 15 minutes designing components and data flow, 10 minutes discussing scaling and trade-offs, 10 minutes for questions and summary.

Explain why event sourcing is useful for auditability and state reconstruction.

Describe how events are stored immutably and published asynchronously.

Discuss the role of snapshots to optimize rebuilding state.

Highlight how CQRS separates command and query responsibilities.

Address concurrency and ordering challenges in event processing.

Mention scaling strategies for event store, message broker, and read models.