Network partitions and split-brain in RabbitMQ - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What happens during a network partition in RabbitMQ?

Imagine a RabbitMQ cluster split into two groups by a network failure. What is the most likely outcome?

A The cluster automatically merges messages without any risk of duplication.

B Both groups continue accepting messages independently, risking data inconsistency (split-brain).

C All nodes stop processing messages until the network is restored.

D Only the master node processes messages; others become read-only.

💡 Hint

Think about what happens when two parts of a cluster can't communicate but still try to work.

During a network partition, RabbitMQ nodes in each partition may continue working independently. This can cause a split-brain scenario where data diverges, risking inconsistency.

💻 Command Output

intermediate

Identify the output of RabbitMQ cluster status during a partition

Given a RabbitMQ cluster with nodes 'node1', 'node2', and 'node3', if 'node3' is isolated by a network partition, what will the rabbitmqctl cluster_status command show on 'node1'?

RabbitMQ

rabbitmqctl cluster_status

A Shows 'node3' as running but with status 'stale'.

B Shows all three nodes as running and connected.

C Shows only 'node1' and 'node2' as running; 'node3' is missing or disconnected.

D Shows an error and no cluster information.

💡 Hint

Consider what happens when a node is unreachable in a cluster status check.

The cluster status command on a node shows only nodes it can communicate with. If 'node3' is partitioned, it won't appear as connected on 'node1'.

❓ Troubleshoot

advanced

Resolving split-brain in RabbitMQ cluster

You notice two partitions in your RabbitMQ cluster, each accepting messages independently. Which step should you take to safely resolve the split-brain?

A Forcefully stop one partition and restart it to rejoin the cluster.

B Restart all nodes simultaneously to reset the cluster state.

C Delete all queues on both partitions to avoid conflicts.

D Ignore the partitions; RabbitMQ will automatically merge data later.

💡 Hint

Think about how to safely restore cluster consistency without losing messages.

Stopping one partition and restarting it allows it to rejoin the main cluster, resolving split-brain safely. Restarting all nodes or deleting queues risks data loss.

✅ Best Practice

advanced

Preventing split-brain in RabbitMQ clusters

Which configuration is best to prevent split-brain scenarios in RabbitMQ clusters?

A Enable the 'pause_minority' partition handling strategy.

B Disable all partition handling to keep nodes always running.

C Use manual node restarts after partitions occur.

D Configure nodes to accept messages regardless of cluster state.

💡 Hint

Consider how RabbitMQ can automatically handle minority partitions.

The 'pause_minority' strategy pauses nodes in the minority partition to avoid split-brain. Other options risk data inconsistency or require manual intervention.

🔀 Workflow

expert

Order the steps to recover a RabbitMQ cluster after a network partition

Arrange the following steps in the correct order to safely recover a RabbitMQ cluster after a network partition causing split-brain.

A 1,3,2,4

B 2,3,1,4

C 2,1,3,4

D 1,2,3,4

💡 Hint

Think about identifying data first, then stopping minority, then restarting.

First, identify the partition with the latest data to preserve it. Then stop the minority partition nodes to avoid conflicts. Restart them to rejoin, and finally verify cluster health.