How to Implement Job Queue Using Redis: Simple Guide

To implement a job queue using Redis, use a list data structure with commands like LPUSH to add jobs and RPOP or BRPOP to retrieve jobs for processing. This creates a simple FIFO queue where workers can safely pull tasks to execute asynchronously.

Syntax

Redis uses list commands to manage a job queue. The main commands are:

• LPUSH queue_name job_data: Adds a job to the start (left) of the queue.
• RPOP queue_name: Removes and returns the last (right) job from the queue.
• BRPOP queue_name timeout: Blocks and waits for a job if the queue is empty, useful for workers.

This setup ensures jobs are processed in the order they were added (FIFO).

redis

LPUSH queue_name job_data
RPOP queue_name
BRPOP queue_name timeout

Example

This example shows how to add jobs to a queue and how a worker can fetch and process them using Redis commands.

redis

# Add jobs to the queue
LPUSH job_queue "job1"
LPUSH job_queue "job2"

# Worker fetches a job (non-blocking)
RPOP job_queue

# Worker waits for a job if queue is empty (blocking with 5 seconds timeout)
BRPOP job_queue 5

Output

"job1"
"job2"
1) "job2"
2) "job1"

Common Pitfalls

Common mistakes when implementing a Redis job queue include:

• Using LPUSH and LPOP together, which reverses job order (LIFO instead of FIFO).
• Not using BRPOP for workers, causing busy waiting and high CPU usage.
• Not handling empty queue cases, leading to errors or missed jobs.
• Not considering job acknowledgment or retries, which Redis lists alone do not provide.

Use LPUSH with RPOP for FIFO and BRPOP for blocking waits to avoid these issues.

redis

Wrong order example:
LPUSH job_queue "job1"
LPOP job_queue  # This returns "job1" if "job1" was pushed last, reversing order

Right order example:
LPUSH job_queue "job1"
RPOP job_queue  # Returns "job1", preserving FIFO

Quick Reference

Command	Description
LPUSH queue_name job_data	Add job to the front of the queue
RPOP queue_name	Remove and get job from the end of the queue (FIFO)
BRPOP queue_name timeout	Block and wait for a job if queue is empty
LLEN queue_name	Get the number of jobs in the queue
LRANGE queue_name start stop	View jobs in the queue

Key Takeaways

Use Redis lists with LPUSH and RPOP to create a FIFO job queue.

Use BRPOP for workers to wait efficiently for new jobs.

Avoid mixing LPUSH with LPOP to prevent reversing job order.

Handle empty queues properly to avoid errors or busy waiting.

Redis lists do not support job acknowledgment; consider additional logic for retries.