Message queue concept in HLD - System Design Exercise

Design: Message Queue System

Design the core message queue system including producers, consumers, queue storage, and delivery mechanisms. Out of scope: detailed UI, security/authentication, and complex routing features.

Functional Requirements

FR1: Allow multiple producers to send messages asynchronously

FR2: Allow multiple consumers to receive and process messages

FR3: Ensure messages are delivered at least once

FR4: Support message ordering within a queue

FR5: Handle message persistence to avoid data loss

FR6: Support message acknowledgment from consumers

FR7: Allow scaling to handle 10,000 messages per second

FR8: Provide low latency with p99 latency under 100ms for message delivery

Non-Functional Requirements

NFR1: System must be highly available with 99.9% uptime

NFR2: Messages must not be lost even if a consumer or producer crashes

NFR3: Support horizontal scaling for producers and consumers

NFR4: Latency target: p99 < 100ms for message delivery

NFR5: Throughput target: 10,000 messages per second

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

Key Components

Producers to send messages

Consumers to receive messages

Message queue storage (in-memory or persistent)

Message broker to manage queues and delivery

Acknowledgment and retry mechanism

Load balancer or API gateway for producers and consumers

Design Patterns

Publish-Subscribe pattern

Work Queue pattern

At-least-once delivery with acknowledgments

Message persistence and durability

Horizontal scaling with partitioned queues

Reference Architecture

  +------------+       +--------------+       +------------+
  |  Producer  | ----> | Message Queue| ----> |  Consumer  |
  +------------+       |   Broker     |       +------------+
                       | (Persistent) |
                       +--------------+
                             ^  |
                             |  v
                      +----------------+
                      |  Message Store  |
                      +----------------+

Components

Producer

Any client application or service

Send messages asynchronously to the message queue broker

Message Queue Broker

Custom or open-source broker (e.g., RabbitMQ, Kafka)

Manage queues, store messages persistently, and deliver messages to consumers

Message Store

Durable storage like disk-based database or log storage

Persist messages to prevent data loss on failures

Consumer

Client applications or services

Receive messages from the broker and process them

Acknowledgment Mechanism

Broker feature

Ensure messages are processed and allow retries if needed

Request Flow

1. Producer sends a message to the message queue broker asynchronously.

2. Broker stores the message persistently in the message store.

3. Broker places the message in the appropriate queue.

4. Consumer subscribes to the queue and receives messages.

5. Consumer processes the message and sends acknowledgment to the broker.

6. Broker marks the message as processed and removes it from the queue.

7. If acknowledgment is not received within timeout, broker retries delivery.

Database Schema

Entities: - Message: id (PK), payload, timestamp, status (pending, delivered, acknowledged), queue_id - Queue: id (PK), name, created_at - Consumer: id (PK), name, subscription details Relationships: - Each Message belongs to one Queue (1:N Queue to Message) - Consumers subscribe to Queues (N:N with subscription table)

Scaling Discussion

Bottlenecks

Message broker CPU and memory limits under high throughput

Disk I/O bottleneck for message persistence

Network bandwidth limits between producers, broker, and consumers

Single queue becoming a hotspot causing uneven load

Consumer processing speed limiting message consumption

Solutions

Partition queues to distribute load across multiple broker instances

Use fast persistent storage like SSDs or distributed log storage

Implement load balancing for producers and consumers

Scale consumers horizontally to increase processing capacity

Use batching and compression to reduce network overhead

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, and 5 minutes summarizing.

Explain asynchronous communication benefits and use cases

Discuss message durability and acknowledgment for reliability

Describe how to handle ordering and retries

Highlight scaling strategies like partitioning and horizontal scaling

Mention trade-offs between latency, throughput, and consistency

Practice

1. What is the primary purpose of a message queue in system design?

easy

A. To speed up the CPU processing directly

B. To permanently save all data for backup

C. To replace the database in storing user information

D. To store messages temporarily until the receiver is ready to process them

Solution

1. Step 1: Understand message queue role

   A message queue acts as a temporary storage for messages between systems or components.

2. Step 2: Identify correct purpose

   It holds messages safely until the receiver can process them, helping with speed differences and reliability.

3. Final Answer:

   To store messages temporarily until the receiver is ready to process them -> Option D

4. Quick Check:

   Message queue = Temporary message storage [OK]

Hint: Message queues hold messages until processing [OK]

Common Mistakes:

• Thinking message queues permanently store data
• Confusing message queues with databases
• Assuming message queues speed up CPU directly

2. Which of the following is the correct way to describe a FIFO queue behavior in a message queue?

easy

A. Messages are processed in random order

B. Messages are processed in the order they arrive

C. Messages are processed starting from the largest size

D. Messages are processed only when the queue is full

Solution

1. Step 1: Define FIFO in message queues

   FIFO means First In, First Out, so messages are handled in the order they arrive.

2. Step 2: Match FIFO behavior to options

   Only Messages are processed in the order they arrive describes processing messages in arrival order, matching FIFO.

3. Final Answer:

   Messages are processed in the order they arrive -> Option B

4. Quick Check:

   FIFO = Process in arrival order [OK]

Hint: FIFO means first message goes out first [OK]

Common Mistakes:

• Confusing FIFO with random or size-based order
• Thinking queue processes only when full
• Mixing FIFO with LIFO (Last In First Out)

3. Consider a message queue system where messages arrive at 10 per second and are processed at 8 per second. What will happen over time?

medium

A. The queue will shrink because processing rate is faster

B. The queue will stay stable because arrival and processing rates are equal

C. The queue will grow indefinitely because arrival rate > processing rate

D. The queue will empty immediately after first message

Solution

1. Step 1: Compare arrival and processing rates

   Messages arrive at 10 per second but only 8 are processed per second.

2. Step 2: Analyze queue growth

   Since arrival rate is higher, messages accumulate, making the queue grow over time.

3. Final Answer:

   The queue will grow indefinitely because arrival rate > processing rate -> Option C

4. Quick Check:

   Arrival > Processing = Queue grows [OK]

Hint: If arrival > processing, queue grows [OK]

Common Mistakes:

• Assuming queue stays stable despite higher arrival rate
• Thinking processing rate is faster than arrival
• Ignoring queue growth over time

4. A message queue system is dropping messages unexpectedly. Which of the following is the most likely cause?

medium

A. The queue size limit is too small and messages are discarded when full

B. The messages are processed faster than they arrive

C. The queue uses FIFO order correctly

D. The system has infinite storage capacity

Solution

1. Step 1: Identify reason for message drops

   Messages drop usually when the queue is full and cannot accept new messages.

2. Step 2: Match cause to options

   The queue size limit is too small and messages are discarded when full states the queue size is too small causing discards, which explains drops.

3. Final Answer:

   The queue size limit is too small and messages are discarded when full -> Option A

4. Quick Check:

   Small queue size = message drops [OK]

Hint: Full queue causes message drops [OK]

Common Mistakes:

• Assuming faster processing causes drops
• Ignoring queue size limits
• Thinking infinite storage exists

5. You need to design a message queue system for a high-traffic online store where orders must be processed reliably and in order. Which design choice best supports this requirement?

hard

A. Use a distributed FIFO queue with message persistence and retry mechanisms

B. Use an in-memory queue without persistence for faster speed

C. Use a random order queue to balance load evenly

D. Use a queue that drops messages when overloaded to avoid delays

Solution

1. Step 1: Understand requirements for reliability and order

   The system must process orders reliably and maintain the order of messages.

2. Step 2: Evaluate design options

   A distributed FIFO queue ensures order, persistence prevents data loss, and retries handle failures.

3. Step 3: Eliminate unsuitable options

   In-memory queues risk data loss, random order breaks order, and dropping messages harms reliability.

4. Final Answer:

   Use a distributed FIFO queue with message persistence and retry mechanisms -> Option A

5. Quick Check:

   Reliable ordered processing = distributed FIFO + persistence [OK]

Hint: Reliable order needs FIFO + persistence + retries [OK]

Common Mistakes:

• Choosing in-memory queues risking data loss
• Ignoring message order importance
• Allowing message drops under load