Kafka vs RabbitMQ vs Redis Pub/Sub - CLI Comparison

Introduction

When you want to send messages between different parts of your system, you need a tool that can handle this smoothly. Kafka, RabbitMQ, and Redis Pub/Sub are three popular tools that help with sending and receiving messages, but they work in different ways and are good for different jobs.

When you need to process a large stream of data reliably and keep a history of messages, like tracking user activity on a website.

When you want to send messages between different services in your app with guaranteed delivery and complex routing, like order processing in an online store.

When you want a simple way to send real-time notifications or updates to many users quickly, like chat messages or live scores.

When you need to handle high message throughput with fault tolerance, such as logging events from many servers.

When you want a lightweight, fast messaging system for simple pub/sub without message persistence.

Commands

This command creates a Kafka topic named 'example-topic' with 3 partitions and a replication factor of 1. Topics are where Kafka stores messages.

Terminal

kafka-topics.sh --create --topic example-topic --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1

Expected Output

Created topic example-topic.

→

--topic - Name of the topic to create

→

--partitions - Number of partitions for parallelism

→

--replication-factor - Number of copies of data for fault tolerance

This command creates a virtual host in RabbitMQ named 'example_vhost' to isolate messaging environments.

Terminal

rabbitmqctl add_vhost example_vhost

Expected Output

No output (command runs silently)

This command sends a message 'Hello subscribers!' to all clients subscribed to 'example-channel' in Redis Pub/Sub.

Terminal

redis-cli PUBLISH example-channel "Hello subscribers!"

Expected Output

(integer) 1

This command reads one message from the beginning of the 'example-topic' in Kafka to verify messages are stored and retrievable.

Terminal

kafka-console-consumer.sh --topic example-topic --bootstrap-server localhost:9092 --from-beginning --max-messages 1

Expected Output

Hello Kafka

Processed a total of 1 messages

→

--from-beginning - Read messages from the start of the topic

→

--max-messages - Limit the number of messages to consume

This command lists all queues in the 'example_vhost' virtual host in RabbitMQ to check message queues.

Terminal

rabbitmqctl list_queues -p example_vhost

Expected Output

Queue	Messages

my-queue	0

→

-p - Specify the virtual host

Key Concept

If you remember nothing else, remember: Kafka is best for high-throughput, durable streaming; RabbitMQ is best for complex routing and guaranteed delivery; Redis Pub/Sub is best for simple, fast, real-time messaging without persistence.

Common Mistakes

Using Redis Pub/Sub when message durability is needed

Redis Pub/Sub does not store messages, so if a subscriber is offline, it misses messages.

Use Kafka or RabbitMQ when you need to keep messages until consumers process them.

Creating too few partitions in Kafka for high load

Too few partitions limit parallel processing and throughput.

Create enough partitions to allow multiple consumers to read in parallel.

Not setting up virtual hosts in RabbitMQ for environment separation

Without virtual hosts, different apps share the same queues and exchanges, causing conflicts.

Use virtual hosts to isolate messaging environments.

Summary

Kafka stores messages durably and handles large data streams with partitions and replication.

RabbitMQ supports complex message routing and guarantees delivery with queues and exchanges.

Redis Pub/Sub offers fast, simple messaging but does not keep messages for offline subscribers.