Consumer groups concept in Redis - Mini Project: Build & Apply

Working with Redis Consumer Groups

📖 Scenario: You are building a simple message processing system using Redis streams. Multiple consumers will read messages from the same stream using a consumer group to share the workload.

🎯 Goal: Create a Redis stream and a consumer group, then read messages from the stream using the consumer group concept.

📋 What You'll Learn

Create a Redis stream called mystream with two messages having fields name and age.

Create a consumer group called mygroup for the stream mystream starting from the beginning of the stream.

Read messages from mystream using the consumer group mygroup and consumer name consumer1.

Acknowledge the messages read by consumer1 to mark them as processed.

💡 Why This Matters

🌍 Real World

Redis streams with consumer groups are used in real-time data processing systems where multiple workers share the workload of processing messages efficiently.

💼 Career

Understanding consumer groups in Redis is valuable for backend developers and system architects building scalable, fault-tolerant message processing pipelines.

1

Create the Redis stream with initial messages

Use the Redis command XADD mystream * to add two messages to the stream mystream. Each message should have fields name and age with these exact values: first message name=Alice, age=30; second message name=Bob, age=25.

Redis

# Add two messages to the Redis stream mystream
# Your code here

Need a hint?

Use XADD mystream * to add messages with fields name and age.

2

Create a consumer group for the stream

Use the Redis command XGROUP CREATE mystream mygroup 0 to create a consumer group called mygroup for the stream mystream starting from the beginning (ID 0).

Redis

XADD mystream * name Alice age 30
XADD mystream * name Bob age 25
# Create the consumer group mygroup starting from 0
# Your code here

Need a hint?

Use XGROUP CREATE mystream mygroup 0 to create the group starting from the first message.

3

Read messages from the stream using the consumer group

Use the Redis command XREADGROUP GROUP mygroup consumer1 COUNT 2 STREAMS mystream > to read up to 2 new messages from the stream mystream as consumer consumer1 in the group mygroup.

Redis

XADD mystream * name Alice age 30
XADD mystream * name Bob age 25
XGROUP CREATE mystream mygroup 0 MKSTREAM
# Read messages as consumer1 from mygroup
# Your code here

Need a hint?

Use XREADGROUP GROUP mygroup consumer1 COUNT 2 STREAMS mystream > to read new messages.

4

Acknowledge the messages after processing

Use the Redis command XACK mystream mygroup followed by the message IDs you read to acknowledge them. For example, if the message IDs are 1526985058136-0 and 1526985058137-0, the command is XACK mystream mygroup 1526985058136-0 1526985058137-0. Add this command with the exact IDs you got from reading the messages.

Redis

XADD mystream * name Alice age 30
XADD mystream * name Bob age 25
XGROUP CREATE mystream mygroup 0 MKSTREAM
XREADGROUP GROUP mygroup consumer1 COUNT 2 STREAMS mystream >
# Acknowledge the messages by their IDs
# Your code here

Need a hint?

Use XACK mystream mygroup <message-id> to acknowledge messages after processing.