Event sourcing in HLD - System Design Exercise

Design: Event Sourcing System

Design covers event storage, event processing, state rebuilding, and read model projection. Out of scope are UI design and specific business domain logic.

Functional Requirements

FR1: Store all changes to application state as a sequence of events.

FR2: Rebuild current state by replaying events.

FR3: Support querying current state efficiently.

FR4: Allow auditing and debugging by inspecting event history.

FR5: Support event versioning and schema evolution.

FR6: Handle concurrent updates safely.

Non-Functional Requirements

NFR1: System must handle 10,000 events per second.

NFR2: Event replay latency for rebuilding state should be under 5 seconds for 1 million events.

NFR3: Availability target of 99.9% uptime.

NFR4: Event storage must be durable and immutable.

NFR5: Support eventual consistency for read models.

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Event Store (append-only log storage)

Command Handler (validates and creates events)

Event Processor (updates read models)

Snapshot Store (to speed up state rebuilding)

Read Model Database (for queries)

Message Broker (for event distribution)

Design Patterns

Event Sourcing pattern

CQRS (Command Query Responsibility Segregation)

Snapshotting

Event Versioning and Upcasting

Idempotent Event Processing

Reference Architecture

Client
  |
  v
Command Handler ---> Event Store ---> Event Processor ---> Read Model DB
                       |                 ^
                       |                 |
                    Snapshot Store ------
                       |
                       v
                 Event Replay

Components

Command Handler

Custom application logic

Receives commands, validates them, and generates new events.

Event Store

Append-only log database (e.g., Apache Kafka, EventStoreDB)

Stores all events immutably in order.

Event Processor

Background worker or stream processor (e.g., Kafka Streams, Apache Flink)

Processes events to update read models and trigger side effects.

Snapshot Store

Key-value store or database (e.g., Redis, Cassandra)

Stores periodic snapshots of state to speed up rebuilding.

Read Model Database

Relational or NoSQL database (e.g., PostgreSQL, MongoDB)

Stores query-optimized views of current state.

Message Broker

Event streaming platform (e.g., Kafka, RabbitMQ)

Distributes events to processors and other services.

Request Flow

1. Client sends a command to the Command Handler.

2. Command Handler validates and creates one or more events.

3. Events are appended to the Event Store in order.

4. Event Processor consumes events from the Event Store or Message Broker.

5. Event Processor updates Read Model Database and optionally creates snapshots.

6. Client queries the Read Model Database for current state.

7. If needed, system rebuilds state by replaying events from Event Store, using snapshots to optimize.

Database Schema

Entities: - Event: {event_id (PK), aggregate_id, event_type, event_data (JSON), timestamp, version} - Snapshot: {snapshot_id (PK), aggregate_id, snapshot_data (JSON), last_event_version, timestamp} - ReadModel: Query-optimized tables depending on domain, updated by event processors. Relationships: - Events belong to an aggregate identified by aggregate_id. - Snapshots correspond to aggregates and represent state at a certain event version.

Scaling Discussion

Bottlenecks

Event Store write throughput limits at very high event rates.

Event replay latency grows with event history size.

Read Model update lag under heavy event load.

Snapshot storage and retrieval overhead.

Handling concurrent commands causing conflicting events.

Solutions

Partition Event Store by aggregate or topic to increase write throughput.

Use snapshotting to reduce event replay time.

Scale Event Processors horizontally with partitioned event streams.

Optimize snapshot frequency balancing storage and replay speed.

Implement optimistic concurrency control and conflict resolution strategies.

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing components and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why event sourcing stores state changes as events.

Describe how event replay and snapshots work together.

Discuss how CQRS separates command and query responsibilities.

Highlight handling of event versioning and schema evolution.

Address scaling challenges and concurrency control.

Practice

1. What is the main idea behind event sourcing in system design?

easy

A. Store all changes as a sequence of events to reconstruct state

B. Store only the latest snapshot of data for quick access

C. Use events only for logging errors in the system

D. Send events to users as notifications without storing them

Solution

1. Step 1: Understand event sourcing concept

   Event sourcing means saving every change as an event, not just the final data.

2. Step 2: Identify how state is managed

   The current state is rebuilt by applying all stored events in order, not by snapshots alone.

3. Final Answer:

   Store all changes as a sequence of events to reconstruct state -> Option A

4. Quick Check:

   Event sourcing = store events to rebuild state [OK]

Hint: Event sourcing saves changes as events, not just snapshots [OK]

Common Mistakes:

• Confusing event sourcing with snapshot-only storage
• Thinking events are only for error logs
• Believing events are just notifications

2. Which of the following is the correct way to represent an event in an event sourcing system?

easy

A. { "eventType": "UserCreated", "timestamp": "2024-06-01T12:00:00Z", "data": { "userId": 123 } }

B. [ "UserCreated", 123, "2024-06-01" ]

C. "UserCreated: userId=123 at 2024-06-01"

D. CREATE USER 123 AT 2024-06-01

Solution

1. Step 1: Identify proper event structure

   Events should be structured data with type, timestamp, and data fields for clarity and processing.

2. Step 2: Compare options

   { "eventType": "UserCreated", "timestamp": "2024-06-01T12:00:00Z", "data": { "userId": 123 } } uses a clear JSON object with eventType, timestamp, and data, which is standard practice.

3. Final Answer:

   { "eventType": "UserCreated", "timestamp": "2024-06-01T12:00:00Z", "data": { "userId": 123 } } -> Option A

4. Quick Check:

   Event = structured JSON with type and data [OK]

Hint: Events are structured objects with type, timestamp, and data [OK]

Common Mistakes:

• Using unstructured strings for events
• Confusing event data with SQL commands
• Using arrays without keys for event details

3. Given these events in order:
[{"eventType":"AddItem","data":{"itemId":1}}, {"eventType":"AddItem","data":{"itemId":2}}, {"eventType":"RemoveItem","data":{"itemId":1}}]
What is the final state of the item list?

medium

A. [1, 2]

B. [2]

C. [1]

D. []

Solution

1. Step 1: Apply events in order to the item list

   Start with empty list. Add item 1 -> [1]. Add item 2 -> [1, 2]. Remove item 1 -> [2].

2. Step 2: Determine final list content

   After all events, only item 2 remains in the list.

3. Final Answer:

   [2] -> Option B

4. Quick Check:

   Apply events sequentially = final list [2] [OK]

Hint: Apply events one by one to get final state [OK]

Common Mistakes:

• Ignoring remove event
• Applying events out of order
• Assuming all added items remain

4. You notice that replaying all events to rebuild state is very slow. What is a common solution to improve performance in event sourcing?

medium

A. Store only the latest event for each entity

B. Delete old events after 1 day to reduce size

C. Use snapshots to save intermediate states periodically

D. Switch to storing only current state, no events

Solution

1. Step 1: Identify performance issue cause

   Replaying all events from the start can be slow as event count grows.

2. Step 2: Choose common optimization

   Snapshots save the full state at points in time, so replay starts from snapshot, reducing replay time.

3. Final Answer:

   Use snapshots to save intermediate states periodically -> Option C

4. Quick Check:

   Snapshots speed up event replay [OK]

Hint: Use snapshots to avoid replaying all events every time [OK]

Common Mistakes:

• Deleting events breaks history and audit
• Keeping only latest event loses full history
• Abandoning events loses event sourcing benefits

5. You design an event sourcing system for a bank. Which approach best ensures data consistency and auditability when multiple transactions happen concurrently?

hard

A. Process events in random order to improve throughput

B. Allow events to overwrite each other without checks for speed

C. Store only final balances without event history to simplify design

D. Use optimistic concurrency control with event versioning and conflict detection

Solution

1. Step 1: Understand concurrency challenges in event sourcing

   Concurrent transactions can cause conflicts if events overwrite each other or are applied out of order.

2. Step 2: Choose method to maintain consistency and audit

   Optimistic concurrency control uses event version numbers to detect conflicts and prevent overwrites, preserving history and correctness.

3. Final Answer:

   Use optimistic concurrency control with event versioning and conflict detection -> Option D

4. Quick Check:

   Optimistic concurrency = safe concurrent event handling [OK]

Hint: Use versioning to detect conflicts in concurrent events [OK]

Common Mistakes:

• Ignoring conflicts causes data corruption
• Dropping event history loses audit trail
• Processing events unordered breaks state correctness