Message brokers (Kafka, RabbitMQ) in Microservices - System Design Exercise

Design: Microservices Communication with Message Brokers

Design focuses on integrating Kafka and RabbitMQ as message brokers for microservices communication. Out of scope are the internal logic of microservices and detailed security implementations.

Functional Requirements

FR1: Enable asynchronous communication between microservices

FR2: Support message durability and reliability

FR3: Handle high throughput of messages (up to 100,000 messages per second)

FR4: Ensure message ordering where required

FR5: Support multiple consumers for the same message stream

FR6: Allow message filtering and routing

FR7: Provide monitoring and alerting for message processing failures

Non-Functional Requirements

NFR1: Latency for message delivery should be under 100ms p99

NFR2: System availability target of 99.9% uptime

NFR3: Support horizontal scaling of message brokers and consumers

NFR4: Ensure at-least-once delivery semantics

NFR5: Handle message retention for up to 7 days

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

❓ Question 7

Key Components

Producers (microservices sending messages)

Message broker cluster (Kafka or RabbitMQ)

Consumers (microservices receiving messages)

Message queues or topics

Message serialization format (e.g., JSON, Avro)

Monitoring and alerting system

Load balancers or proxies for broker access

Design Patterns

Publish-Subscribe pattern

Message Queue pattern

Event Sourcing

Dead Letter Queue for failed messages

Consumer Group for load balancing

Message filtering and routing

Reference Architecture

 +-------------+       +----------------+       +-------------+
 |  Producer   | ----> |  Message Broker | ----> |  Consumer   |
 | (Service A) |       | (Kafka/Rabbit) |       | (Service B) |
 +-------------+       +----------------+       +-------------+
         |                     |                      |
         |                     |                      |
         |                     v                      |
         |             +----------------+             |
         |             |  Monitoring &   |             |
         |             |   Alerting     |             |
         |             +----------------+             |
         |                                         
         +-----------------------------------------+
                         Message Flow

Components

Producer

Microservice (any language)

Sends messages/events to the message broker asynchronously

Message Broker Cluster

Apache Kafka or RabbitMQ

Stores, routes, and delivers messages reliably between producers and consumers

Consumer

Microservice (any language)

Receives and processes messages from the broker

Monitoring & Alerting

Prometheus + Grafana or equivalent

Tracks message broker health, message lag, failures, and alerts on issues

Request Flow

1. Producer microservice creates a message and serializes it (e.g., JSON).

2. Producer sends the message to the message broker topic or queue.

3. Message broker stores the message durably and routes it to subscribed consumers.

4. Consumers in a consumer group receive messages for processing.

5. If processing succeeds, consumer commits the message offset (Kafka) or acknowledges (RabbitMQ).

6. If processing fails, message can be retried or sent to a dead letter queue.

7. Monitoring system collects metrics on message throughput, lag, and failures.

8. Alerts are triggered if message processing falls behind or errors increase.

Database Schema

Not applicable as message brokers use internal storage. Key entities: Topics (Kafka) or Queues (RabbitMQ), Messages with metadata (offset, timestamp, key), Consumer Groups for load balancing.

Scaling Discussion

Bottlenecks

Message broker storage and throughput limits under high load

Consumer processing speed causing message backlog

Network bandwidth between producers, brokers, and consumers

Single point of failure if broker cluster is not highly available

Monitoring system overload with large metrics volume

Solutions

Scale broker cluster horizontally by adding more nodes and partitions (Kafka) or nodes and queues (RabbitMQ)

Increase number of consumer instances in consumer groups to parallelize processing

Use compression and efficient serialization to reduce message size

Deploy brokers in a highly available cluster with replication and failover

Implement sampling or aggregation in monitoring to reduce metrics volume

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why asynchronous messaging is needed for microservices decoupling

Discuss differences between Kafka and RabbitMQ and when to use each

Describe message delivery guarantees and how to handle failures

Show understanding of consumer groups and load balancing

Highlight monitoring importance and how to detect issues early

Address scaling challenges and practical solutions

Practice

1. What is the primary role of a message broker like Kafka or RabbitMQ in a microservices architecture?

easy

A. To store large amounts of user data permanently

B. To enable services to communicate asynchronously by passing messages

C. To replace the database in microservices

D. To directly execute business logic in services

Solution

1. Step 1: Understand message broker function

   Message brokers act as middlemen that help services send and receive messages without waiting for each other.

2. Step 2: Identify correct role in microservices

   They enable asynchronous communication, improving scalability and fault tolerance.

3. Final Answer:

   To enable services to communicate asynchronously by passing messages -> Option B

4. Quick Check:

   Message broker = asynchronous communication [OK]

Hint: Message brokers pass messages between services asynchronously [OK]

Common Mistakes:

• Confusing brokers with databases
• Thinking brokers execute business logic
• Assuming brokers store permanent user data

2. Which of the following is the correct way to declare a RabbitMQ queue in code?

easy

A. channel.queueDeclare('task_queue', true, false, false, null);

B. channel.createQueue('task_queue', durable=true);

C. queue.declare('task_queue', persistent=True);

D. rabbitmq.queue('task_queue', durable=True);

Solution

1. Step 1: Recall RabbitMQ queue declaration syntax

   In RabbitMQ Java client, channel.queueDeclare is used with parameters: queue name, durable, exclusive, autoDelete, and arguments.

2. Step 2: Match correct syntax

   channel.queueDeclare('task_queue', true, false, false, null); matches the official method signature and parameter order correctly.

3. Final Answer:

   channel.queueDeclare('task_queue', true, false, false, null); -> Option A

4. Quick Check:

   RabbitMQ queueDeclare syntax = channel.queueDeclare('task_queue', true, false, false, null); [OK]

Hint: Remember RabbitMQ uses channel.queueDeclare with 5 parameters [OK]

Common Mistakes:

• Using incorrect method names like createQueue
• Passing parameters with wrong names or order
• Confusing RabbitMQ syntax with other brokers

3. Given the following Kafka consumer code snippet, what will be the output if the topic has 3 messages and auto-commit is enabled?

consumer.subscribe(['orders'])
for message in consumer.poll(timeout_ms=1000).values():
    print(message.value.decode('utf-8'))

medium

A. Prints nothing because poll returns a dict of lists

B. Prints only the first message and stops

C. Prints all 3 messages from the 'orders' topic

D. Raises an error due to wrong method usage

Solution

1. Step 1: Analyze Kafka consumer.poll() return type

   The poll() method returns a dictionary where keys are partitions and values are lists of messages.

2. Step 2: Understand iteration over poll().values()

   Iterating over values() gives lists of messages, not individual messages, so calling message.value will cause an error because message is a list, not a message object.

3. Final Answer:

   Raises an error due to wrong method usage -> Option D

4. Quick Check:

   poll() returns dict of lists; iterating directly over values and accessing message.value causes error [OK]

Hint: poll() returns dict of lists, not single messages [OK]

Common Mistakes:

• Assuming poll() returns a flat list of messages
• Not decoding message values properly
• Ignoring that poll() returns per-partition batches

4. A developer wrote this RabbitMQ consumer code but it never receives messages:

channel.basicConsume('task_queue', autoAck=False, callback=process_message)

What is the likely issue?

medium

A. The consumer must call channel.start_consuming() to begin receiving messages

B. The callback function name should be 'on_message' instead of 'process_message'

C. autoAck must be set to True for messages to be received

D. The queue name 'task_queue' is invalid and must be changed

Solution

1. Step 1: Understand RabbitMQ consumer lifecycle

   After setting up basicConsume, the consumer must start the event loop with channel.start_consuming() to receive messages.

2. Step 2: Identify missing call

   The code lacks start_consuming(), so no messages are delivered.

3. Final Answer:

   The consumer must call channel.start_consuming() to begin receiving messages -> Option A

4. Quick Check:

   Missing start_consuming() = The consumer must call channel.start_consuming() to begin receiving messages [OK]

Hint: Remember to call start_consuming() after basicConsume [OK]

Common Mistakes:

• Thinking callback function name must be fixed
• Believing autoAck controls message receipt
• Assuming queue name is invalid without evidence

5. You need to design a scalable order processing system using Kafka. Which approach best ensures message order per customer while allowing parallel processing across customers?

hard

A. Use a single Kafka partition for all orders to keep global order

B. Use multiple topics, one per customer, to isolate order streams

C. Partition messages by customer ID so each customer's orders stay ordered in their partition

D. Send all orders to a single consumer instance to maintain order

Solution

1. Step 1: Understand Kafka partitioning and ordering

   Kafka guarantees order only within a partition, so to keep order per customer, messages must be partitioned by customer ID.

2. Step 2: Evaluate options for scalability and ordering

   Partitioning by customer ID allows parallel processing across partitions (customers) while preserving order per customer.

3. Final Answer:

   Partition messages by customer ID so each customer's orders stay ordered in their partition -> Option C

4. Quick Check:

   Partition by key for order + parallelism = Partition messages by customer ID so each customer's orders stay ordered in their partition [OK]

Hint: Partition by customer ID to keep order and scale processing [OK]

Common Mistakes:

• Using single partition limits scalability
• Creating many topics adds unnecessary complexity
• Using single consumer blocks parallelism