Kafka vs RabbitMQ vs Redis Pub/Sub - Interactive Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a Kafka producer that sends a message.

Kafka

from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='localhost:9092')
producer.send('my_topic', [1])

Drag options to blanks, or click blank then click option'

AHello, Kafka!

B'Hello, Kafka!'

Cb'Hello, Kafka!'

Dbytes('Hello, Kafka!', 'utf-8')

Attempts:

3 left

💡 Hint

Common Mistakes

Sending the message as a plain string without converting to bytes.

The Kafka producer expects the message in bytes format, so prefixing the string with b converts it to bytes.

2fill in blank

medium

Complete the code to consume messages from a Kafka topic.

Kafka

from kafka import KafkaConsumer
consumer = KafkaConsumer('my_topic', bootstrap_servers='localhost:9092')
for message in [1]:
    print(message.value)

Drag options to blanks, or click blank then click option'

Aconsumer.poll()

Bconsumer.consume()

Cconsumer.read()

Dconsumer

Attempts:

3 left

💡 Hint

Common Mistakes

Trying to call non-existent methods like consume() or read().

The KafkaConsumer object itself is iterable, so you can loop directly over it to get messages.

3fill in blank

hard

Fix the error in the Kafka producer code to send a JSON message.

Kafka

import json
from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='localhost:9092', value_serializer=lambda v: [1])
producer.send('my_topic', {'key': 'value'})

Drag options to blanks, or click blank then click option'

Ajson.dumps(v).encode('utf-8')

Bjson.loads(v).encode('utf-8')

Cstr(v).encode('utf-8')

Djson.dumps(v)

Attempts:

3 left

💡 Hint

Common Mistakes

Using json.loads which parses JSON instead of serializing.

The value_serializer should convert the Python object to a JSON string and then encode it to bytes.

4fill in blank

hard

Fill both blanks to create a Kafka consumer that reads from the beginning and auto-commits offsets.

Kafka

from kafka import KafkaConsumer
consumer = KafkaConsumer('my_topic', bootstrap_servers='localhost:9092', auto_offset_reset=[1], enable_auto_commit=[2])

Drag options to blanks, or click blank then click option'

A'earliest'

BTrue

C'latest'

DFalse

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'latest' which reads only new messages.

Setting auto_offset_reset to 'earliest' makes the consumer read from the beginning if no offset is found. enable_auto_commit=True enables automatic offset commits.

5fill in blank

hard

Fill all three blanks to create a Kafka producer with a custom serializer and send a message.

Kafka

import json
from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='localhost:9092', value_serializer=[1])
message = {'event': 'login', 'user': 'alice'}
producer.send('events', [2])
producer.flush()  # [3]

Drag options to blanks, or click blank then click option'

Alambda v: json.dumps(v).encode('utf-8')

Bmessage

CEnsure all messages are sent before exiting

Dmessage.encode('utf-8')

Attempts:

3 left

💡 Hint

Common Mistakes

Trying to encode the message twice or forgetting to flush.

The value_serializer converts the message to JSON bytes. The message is sent as is. flush() makes sure all messages are sent before the program ends.

Practice

1. Which messaging system is best suited for storing message history and handling large data streams?

easy

A. None of the above

B. RabbitMQ

C. Redis Pub/Sub

D. Kafka

Solution

1. Step 1: Understand Kafka's purpose

   Kafka is designed to handle large data streams and keeps a durable log of messages, allowing message history to be stored.

2. Step 2: Compare with RabbitMQ and Redis Pub/Sub

   RabbitMQ focuses on reliable delivery and routing, while Redis Pub/Sub is fast but does not store messages.

3. Final Answer:

   Kafka -> Option D

4. Quick Check:

   Message history and big data = Kafka [OK]

Hint: Remember: Kafka stores history; Redis Pub/Sub does not [OK]

Common Mistakes:

• Confusing Redis Pub/Sub as storing messages
• Thinking RabbitMQ stores message history
• Assuming all systems keep message logs

2. Which of the following is the correct statement about Redis Pub/Sub?

easy

A. Redis Pub/Sub guarantees message delivery even if subscribers are offline.

B. Redis Pub/Sub does not store messages; it only delivers messages to active subscribers.

C. Redis Pub/Sub supports complex message routing like RabbitMQ.

D. Redis Pub/Sub is designed for batch processing of messages.

Solution

1. Step 1: Analyze Redis Pub/Sub behavior

   Redis Pub/Sub delivers messages only to subscribers currently connected; it does not store messages for offline clients.

2. Step 2: Check other options

   Redis Pub/Sub guarantees message delivery even if subscribers are offline. is false because Redis Pub/Sub does not guarantee delivery if subscribers are offline. Redis Pub/Sub supports complex message routing like RabbitMQ. is false because Redis Pub/Sub lacks complex routing. Redis Pub/Sub is designed for batch processing of messages. is false because Redis Pub/Sub is for real-time messaging, not batch processing.

3. Final Answer:

   Redis Pub/Sub does not store messages; it only delivers messages to active subscribers. -> Option B

4. Quick Check:

   Redis Pub/Sub delivers only to active subscribers [OK]

Hint: Redis Pub/Sub = no message storage, only live delivery [OK]

Common Mistakes:

• Assuming Redis Pub/Sub stores messages
• Confusing Redis with RabbitMQ routing features
• Thinking Redis Pub/Sub supports offline message delivery

3. Given the following scenario: You need a messaging system that supports flexible routing and guarantees message delivery even if consumers are temporarily offline. Which system fits best?

medium

A. Kafka

B. Redis Pub/Sub

C. RabbitMQ

D. None of the above

Solution

1. Step 1: Understand RabbitMQ features

   RabbitMQ supports flexible routing through exchanges and queues and guarantees message delivery with acknowledgments and persistence.

2. Step 2: Compare with Kafka and Redis Pub/Sub

   Kafka stores messages but is less flexible in routing. Redis Pub/Sub is fast but does not guarantee delivery if consumers are offline.

3. Final Answer:

   RabbitMQ -> Option C

4. Quick Check:

   Flexible routing + guaranteed delivery = RabbitMQ [OK]

Hint: Flexible routing and delivery guarantees point to RabbitMQ [OK]

Common Mistakes:

• Choosing Kafka for flexible routing
• Assuming Redis Pub/Sub guarantees delivery
• Ignoring RabbitMQ's message acknowledgment feature

4. You have a system using Redis Pub/Sub, but messages are lost when subscribers disconnect. What is the best way to fix this issue?

medium

A. Switch to Kafka to store messages and allow offline consumption.

B. Increase Redis server memory to store more messages.

C. Use Redis Pub/Sub with message persistence enabled.

D. Configure Redis to retry sending messages to offline subscribers.

Solution

1. Step 1: Identify Redis Pub/Sub limitation

   Redis Pub/Sub does not store messages; if subscribers disconnect, messages are lost.

2. Step 2: Choose a system that stores messages

   Kafka stores messages durably, allowing consumers to read messages even after being offline.

3. Final Answer:

   Switch to Kafka to store messages and allow offline consumption. -> Option A

4. Quick Check:

   Redis Pub/Sub no storage; Kafka stores messages [OK]

Hint: Redis Pub/Sub can't store messages; Kafka can [OK]

Common Mistakes:

• Trying to enable persistence on Redis Pub/Sub (not supported)
• Assuming Redis memory increase solves message loss
• Expecting Redis to retry offline delivery automatically

5. You want to build a real-time analytics system that processes large volumes of data and keeps a full history of all events for auditing. Which combination of messaging systems is the best approach?

hard

A. Use Kafka for event storage and Redis Pub/Sub for real-time notifications.

B. Use RabbitMQ for event storage and Redis Pub/Sub for message routing.

C. Use Redis Pub/Sub alone for both storage and real-time processing.

D. Use RabbitMQ alone for storing large data streams and real-time analytics.

Solution

1. Step 1: Identify requirements

   The system needs to handle large data volumes, keep full event history, and provide real-time notifications.

2. Step 2: Match systems to requirements

   Kafka is ideal for storing large data streams and keeping history. Redis Pub/Sub is fast for real-time notifications but does not store data.

3. Step 3: Evaluate other options

   RabbitMQ is good for routing and delivery but less suited for big data storage. Redis Pub/Sub alone cannot store messages. RabbitMQ alone is not optimized for large data streams.

4. Final Answer:

   Use Kafka for event storage and Redis Pub/Sub for real-time notifications. -> Option A

5. Quick Check:

   Kafka stores history + Redis Pub/Sub fast notifications [OK]

Hint: Combine Kafka storage with Redis Pub/Sub speed [OK]

Common Mistakes:

• Using Redis Pub/Sub alone for storage
• Assuming RabbitMQ handles big data storage well
• Ignoring Kafka's strength in event history