Kafka vs RabbitMQ vs Redis Pub/Sub - Hands-On Comparison

Kafka vs RabbitMQ vs Redis Pub/Sub Comparison

📖 Scenario: You are working on a messaging system for a small online store. You want to understand how three popular messaging tools work: Kafka, RabbitMQ, and Redis Pub/Sub. Each tool helps different parts of your store talk to each other by sending messages.

🎯 Goal: Build simple examples to see how Kafka, RabbitMQ, and Redis Pub/Sub send and receive messages. This will help you understand their differences and when to use each one.

📋 What You'll Learn

Create a Kafka topic and send a message

Set up a RabbitMQ queue and send a message

Use Redis Pub/Sub to publish and subscribe to a message

Print the received messages from each system

💡 Why This Matters

🌍 Real World

Messaging systems help different parts of an application talk to each other smoothly, like sending orders from a website to a warehouse system.

💼 Career

Understanding Kafka, RabbitMQ, and Redis Pub/Sub is useful for backend developers, system architects, and anyone working with real-time data or distributed systems.

1

Set up Kafka topic and send a message

Create a Kafka producer that sends the message 'Hello Kafka' to the topic 'store-messages'. Use the variable producer for the Kafka producer and topic for the topic name.

Kafka

# Create Kafka producer and send 'Hello Kafka' to 'store-messages'
# Your code here

Need a hint?

Use KafkaProducer from the kafka library. The message must be bytes, so prefix the string with b.

2

Set up RabbitMQ queue and send a message

Create a RabbitMQ connection and channel. Declare a queue named 'store-queue' and send the message 'Hello RabbitMQ' to this queue. Use variables connection, channel, and queue_name.

Kafka

from kafka import KafkaProducer
import pika

producer = KafkaProducer(bootstrap_servers='localhost:9092')
topic = 'store-messages'

producer.send(topic, b'Hello Kafka')
producer.flush()

# Create RabbitMQ connection and send 'Hello RabbitMQ' to 'store-queue'
# Your code here

Need a hint?

Use pika.BlockingConnection to connect to RabbitMQ. Declare the queue before sending the message.

3

Use Redis Pub/Sub to publish and subscribe

Create a Redis client using redis.Redis(). Subscribe to the channel 'store-channel' and publish the message 'Hello Redis' to it. Use variables redis_client and channel_name.

Kafka

from kafka import KafkaProducer
import pika
import redis

producer = KafkaProducer(bootstrap_servers='localhost:9092')
topic = 'store-messages'

producer.send(topic, b'Hello Kafka')
producer.flush()

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
queue_name = 'store-queue'
channel.queue_declare(queue=queue_name)
channel.basic_publish(exchange='', routing_key=queue_name, body='Hello RabbitMQ')
connection.close()

# Create Redis client, subscribe to 'store-channel', and publish 'Hello Redis'
# Your code here

Need a hint?

Use redis.Redis() to connect. Use pubsub.subscribe() to listen and publish() to send messages.

4

Print received messages from Kafka, RabbitMQ, and Redis

Print the messages received from Kafka topic 'store-messages', RabbitMQ queue 'store-queue', and Redis channel 'store-channel'. Use variables producer, connection, channel, and redis_client as needed. Show the exact messages: Hello Kafka, Hello RabbitMQ, and Hello Redis.

Kafka

from kafka import KafkaProducer
import pika
import redis

producer = KafkaProducer(bootstrap_servers='localhost:9092')
topic = 'store-messages'

producer.send(topic, b'Hello Kafka')
producer.flush()

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
queue_name = 'store-queue'
channel.queue_declare(queue=queue_name)
channel.basic_publish(exchange='', routing_key=queue_name, body='Hello RabbitMQ')
connection.close()

redis_client = redis.Redis()
channel_name = 'store-channel'

pubsub = redis_client.pubsub()
pubsub.subscribe(channel_name)
redis_client.publish(channel_name, 'Hello Redis')

# Print the messages received from Kafka, RabbitMQ, and Redis
# Your code here

Need a hint?

Use KafkaConsumer to read from Kafka. Use channel.basic_get to get RabbitMQ message. Use pubsub.listen() to get Redis message. Decode bytes to strings before printing.