Consumer groups concept in Redis

Introduction

Consumer groups help multiple clients share the work of reading messages from a stream without missing any messages or reading the same message twice.

When you want several workers to process messages from a stream in parallel.

When you need to make sure each message is processed only once by one worker.

When you want to balance the load of message processing across multiple clients.

When you want to track which messages have been processed and which are still pending.

When you want to build a reliable message queue system using Redis streams.

Syntax

Redis

XGROUP CREATE <stream> <groupname> <id or $>
XREADGROUP GROUP <groupname> <consumername> COUNT <count> STREAMS <stream> >
XACK <stream> <groupname> <id> [<id> ...]

XGROUP CREATE creates a new consumer group for a stream.

XREADGROUP reads messages for a specific consumer in the group, marking them as pending until acknowledged.

Examples

Create a consumer group named 'mygroup' on stream 'mystream' starting from new messages.

Redis

XGROUP CREATE mystream mygroup $

Consumer 'consumer1' reads up to 2 new messages from 'mystream' in group 'mygroup'.

Redis

XREADGROUP GROUP mygroup consumer1 COUNT 2 STREAMS mystream >

Acknowledge that message with ID '1526985058136-0' was processed by the group.

Redis

XACK mystream mygroup 1526985058136-0

Sample Program

This example creates a group, adds two messages, reads them as consumer1, then acknowledges them.

Redis

XADD mystream * name Alice
XADD mystream * name Bob
XGROUP CREATE mystream mygroup $ 
XREADGROUP GROUP mygroup consumer1 COUNT 2 STREAMS mystream >
XACK mystream mygroup 1-0 2-0

Important Notes

Consumer groups allow multiple consumers to share the workload without duplicating message processing.

Messages remain pending until acknowledged with XACK, so you can track unprocessed messages.

Use '>' in XREADGROUP to get new messages that were never delivered to any consumer.

Summary

Consumer groups let many clients read from the same stream safely and efficiently.

They help avoid duplicate processing by tracking message delivery and acknowledgment.

You create groups, read messages as consumers, and acknowledge messages when done.

Practice

1. What is the main purpose of Redis consumer groups in streams?

easy

A. Allow multiple clients to read the same stream without duplicating work

B. Store data permanently in Redis

C. Encrypt messages in the stream

D. Automatically delete old messages from the stream

Solution

1. Step 1: Understand consumer groups role

   Consumer groups let many clients read from the same stream safely and efficiently.

2. Step 2: Identify main benefit

   They avoid duplicate processing by tracking which messages each consumer has read.

3. Final Answer:

   Allow multiple clients to read the same stream without duplicating work -> Option A

4. Quick Check:

   Consumer groups = safe multi-client reading [OK]

Hint: Consumer groups = shared reading without duplicates [OK]

Common Mistakes:

• Thinking consumer groups store data permanently
• Confusing consumer groups with encryption
• Assuming consumer groups delete old messages automatically

2. Which Redis command is used to create a consumer group for a stream named mystream?

easy

A. XGROUP NEW mystream mygroup $

B. XGROUP CREATE mystream mygroup $

C. XGROUP ADD mystream mygroup $

D. XREADGROUP CREATE mystream mygroup $

Solution

1. Step 1: Recall command to create consumer group

   The correct command is XGROUP CREATE followed by stream name, group name, and ID.

2. Step 2: Match command syntax

   XGROUP CREATE mystream mygroup $ correctly creates group mygroup on mystream.

3. Final Answer:

   XGROUP CREATE mystream mygroup $ -> Option B

4. Quick Check:

   Create group = XGROUP CREATE [OK]

Hint: Use XGROUP CREATE to make groups [OK]

Common Mistakes:

• Using XREADGROUP instead of XGROUP CREATE
• Using non-existent commands like XGROUP ADD or XGROUP NEW
• Mixing command order or missing parameters

3. Given a consumer group mygroup on stream mystream, what does this command return?

XREADGROUP GROUP mygroup consumer1 COUNT 2 STREAMS mystream > 

medium

A. Returns up to 2 new messages never delivered to any consumer

B. Returns all messages in the stream from the beginning

C. Returns the last 2 messages already acknowledged

D. Returns an error because of wrong syntax

Solution

1. Step 1: Understand XREADGROUP with '>' ID

   Using '>' means read only new messages never delivered to any consumer in the group.

2. Step 2: COUNT 2 limits messages returned

   The command returns up to 2 such new messages for consumer1.

3. Final Answer:

   Returns up to 2 new messages never delivered to any consumer -> Option A

4. Quick Check:

   XREADGROUP with '>' = new messages only [OK]

Hint: '>' ID in XREADGROUP means new messages only [OK]

Common Mistakes:

• Thinking it returns all messages from start
• Assuming it returns acknowledged messages
• Believing the command syntax is incorrect

4. You run this command but get an error:

XREADGROUP GROUP mygroup consumer1 STREAMS mystream >

What is the likely cause?

medium

A. Missing COUNT argument causes syntax error

B. The '>' symbol is invalid here

C. The command is missing the ID after STREAMS

D. Consumer group 'mygroup' does not exist

Solution

1. Step 1: Verify command syntax

   The syntax is correct: XREADGROUP GROUP mygroup consumer1 STREAMS mystream > uses '>' ID for new messages. COUNT is optional.

2. Step 2: Identify prerequisite error

   XREADGROUP requires an existing consumer group created with XGROUP CREATE. If missing, it returns a NOGROUP error.

3. Final Answer:

   Consumer group 'mygroup' does not exist -> Option D

4. Quick Check:

   XREADGROUP needs existing group [OK]

Hint: XREADGROUP requires existing consumer group [OK]

Common Mistakes:

• Thinking COUNT is mandatory (it's optional)
• Believing '>' is invalid
• Mistakenly thinking ID is missing after STREAMS

5. You have a stream orders and consumer group ordergroup. You want to ensure that if a consumer crashes before acknowledging messages, another consumer can claim and process those pending messages. Which Redis command helps you do this?

hard

A. XREADGROUP GROUP ordergroup consumer2 STREAMS orders >

B. XACK orders ordergroup 1526985058136

C. XCLAIM orders ordergroup consumer2 60000 1526985058136

D. XGROUP CREATE orders ordergroup $

Solution

1. Step 1: Understand the problem of crashed consumers

   If a consumer crashes, its pending messages remain unacknowledged and need to be claimed by others.

2. Step 2: Identify command to claim pending messages

   XCLAIM lets another consumer take ownership of pending messages after a timeout.

3. Step 3: Match command usage

   XCLAIM orders ordergroup consumer2 60000 1526985058136 claims message ID 1526985058136 for consumer2.

4. Final Answer:

   XCLAIM orders ordergroup consumer2 60000 1526985058136 -> Option C

5. Quick Check:

   Claim pending messages = XCLAIM [OK]

Hint: Use XCLAIM to recover unacknowledged messages [OK]

Common Mistakes:

• Using XACK which only acknowledges, not claims
• Using XREADGROUP with '>' which reads new messages only
• Trying to create group again instead of claiming