Task queue concept in Flask - Step-by-Step Execution

Concept Flow - Task queue concept

Client sends request

↓

Add task to queue

↓

Worker picks task

↓

Worker processes task

↓

Worker updates status/result

↓

Client checks result/status

This flow shows how a task is added to a queue, processed by a worker, and the result is updated for the client.

Execution Sample

Flask

from flask import Flask
from rq import Queue
from redis import Redis

app = Flask(__name__)
redis_conn = Redis()
queue = Queue(connection=redis_conn)

def background_task(n):
    return n * 2

@app.route('/start/<int:n>')
def start_task(n):
    job = queue.enqueue(background_task, n)
    return f'Task {job.id} started'

This Flask app adds a background task to a Redis queue and returns the task ID.

Execution Table

Step	Action	Queue State	Worker State	Output/Result
1	Client calls /start/5	Empty	Idle	Task added to queue with ID 1
2	Task 1 enqueued	Task 1 (background_task(5))	Idle	Waiting for worker
3	Worker picks Task 1	Empty	Processing Task 1	Task 1 started
4	Worker runs background_task(5)	Empty	Processing Task 1	Calculating 5 * 2
5	Worker finishes Task 1	Empty	Idle	Result 10 stored
6	Client can check result	Empty	Idle	Result 10 available

💡 Task processed and result stored; queue is empty again.

Variable Tracker

Variable	Start	After Step 1	After Step 3	After Step 5	Final
queue	Empty	Task 1 added	Task 1 removed	Empty	Empty
worker_state	Idle	Idle	Processing Task 1	Idle	Idle
task_result	None	None	None	10	10

Key Moments - 2 Insights

Why does the client get a task ID immediately before the task finishes?

Because the task is added to the queue instantly (see Step 2), the client receives the ID right away while the worker processes it later (Step 3-5).

What happens if the worker is busy when a new task is added?

The new task stays in the queue (Step 2) until the worker finishes current tasks and picks it up (Step 3).

Visual Quiz - 3 Questions

Look at the execution table, what is the worker state at Step 4?

A Idle

B Processing Task 1

C Waiting for task

D Task completed

💡 Hint

Check the 'Worker State' column at Step 4 in the execution table.

At which step does the queue become empty again?

A Step 2

B Step 3

C Step 5

D Step 6

💡 Hint

Look at the 'Queue State' column to see when tasks are removed.

If the client sends two tasks quickly, how would the queue state change at Step 2?

A Queue has two tasks waiting

B Queue is empty

C Worker processes both tasks immediately

D Queue has one task only

💡 Hint

Refer to the queue behavior in the variable_tracker and execution_table for multiple tasks.

Concept Snapshot

Task queue lets Flask handle long jobs outside request time.
Client adds task to queue, gets task ID immediately.
Worker runs tasks one by one from queue.
Results stored separately for client to check later.
This keeps app responsive and scalable.

Full Transcript

In Flask, a task queue helps run long jobs in the background. When a client sends a request, the app adds the job to a queue and returns a task ID right away. A worker process picks tasks from the queue and runs them separately. This way, the app stays fast and doesn't wait for the job to finish. The worker stores the result, which the client can check later. The execution table shows each step: adding the task, worker picking it, processing, and finishing. Variables like queue state and worker state change as tasks move through the system. This method is common to keep web apps responsive when doing heavy work.

Practice

1. What is the main purpose of using a task queue in a Flask application?

easy

A. To serve static files faster

B. To run slow or long tasks in the background without blocking the main app

C. To handle user authentication securely

D. To speed up the Flask app by caching all responses

Solution

1. Step 1: Understand task queue role

   Task queues let slow jobs run separately from the main app, so the app stays responsive.

2. Step 2: Identify the correct purpose

   Running slow tasks in the background matches the main purpose of task queues.

3. Final Answer:

   To run slow or long tasks in the background without blocking the main app -> Option B

4. Quick Check:

   Task queue purpose = background slow jobs [OK]

Hint: Task queues handle slow jobs outside main app flow [OK]

Common Mistakes:

• Confusing task queues with caching
• Thinking task queues handle authentication
• Assuming task queues speed up static file serving

2. Which of the following is the correct way to enqueue a task using rq in Flask?

easy

A. queue.enqueue(my_task, arg1, arg2)

B. queue.run(my_task, arg1, arg2)

C. queue.start(my_task, arg1, arg2)

D. queue.execute(my_task, arg1, arg2)

Solution

1. Step 1: Recall rq enqueue method

   In rq, tasks are added to the queue using the enqueue() method.

2. Step 2: Match method to options

   Only queue.enqueue(my_task, arg1, arg2) uses enqueue() correctly with task and arguments.

3. Final Answer:

   queue.enqueue(my_task, arg1, arg2) -> Option A

4. Quick Check:

   rq enqueue method = enqueue() [OK]

Hint: Use enqueue() to add tasks to rq queue [OK]

Common Mistakes:

• Using run(), start(), or execute() instead of enqueue()
• Forgetting to pass task arguments
• Calling enqueue on wrong object

3. Given this Flask rq task code snippet:

from rq import Queue
from redis import Redis

redis_conn = Redis()
queue = Queue(connection=redis_conn)

def add(x, y):
    return x + y

job = queue.enqueue(add, 3, 4)
print(job.result)

What will be printed immediately after running this code?

medium

A. None

B. 7

C. Error: job.result not available

D. 3 + 4

Solution

1. Step 1: Understand rq job processing

   When a job is enqueued, it runs asynchronously. The result is not ready immediately.

2. Step 2: Check job.result value right after enqueue

   Immediately after enqueue, job.result is None because the worker hasn't processed it yet.

3. Final Answer:

   None -> Option A

4. Quick Check:

   job.result before worker runs = None [OK]

Hint: job.result is None until worker finishes task [OK]

Common Mistakes:

• Assuming job.result is ready immediately
• Expecting synchronous return value
• Confusing job.result with function return

4. You wrote this code to enqueue a task in Flask with rq:

queue.enqueue(my_task, arg1, arg2)

But the task never runs. What is the most likely reason?

medium

A. Redis server is not installed

B. The task function my_task is missing

C. You forgot to start the rq worker process

D. You used enqueue instead of run

Solution

1. Step 1: Check rq worker role

   rq worker must be running to process tasks from the queue.

2. Step 2: Identify missing worker

   If tasks never run, the worker process is likely not started.

3. Final Answer:

   You forgot to start the rq worker process -> Option C

4. Quick Check:

   Worker must run to process tasks [OK]

Hint: Always run rq worker to process queued tasks [OK]

Common Mistakes:

• Assuming enqueue runs task immediately
• Not starting rq worker process
• Confusing Redis installation with worker running

5. You want to run a slow image processing function in the background using Flask and rq. Which combination correctly sets up the task queue and worker?

hard

A. Install Redis, but only enqueue tasks; worker runs automatically without command

B. Create Flask app, call task function directly; no need for Redis or worker

C. Use Flask's built-in threading to run task; no Redis or rq needed

D. Create Redis connection, create rq Queue with it, enqueue task; run rq worker in terminal

Solution

1. Step 1: Setup Redis and rq Queue

   You must create a Redis connection and pass it to rq Queue to enqueue tasks properly.

2. Step 2: Run rq worker separately

   The rq worker must be started manually in the terminal to process queued tasks.

3. Final Answer:

   Create Redis connection, create rq Queue with it, enqueue task; run rq worker in terminal -> Option D

4. Quick Check:

   Redis + rq Queue + worker process = correct setup [OK]

Hint: Redis + rq Queue + manual worker run = background tasks [OK]

Common Mistakes:

• Calling task function directly instead of enqueueing
• Assuming worker runs automatically
• Using Flask threading instead of task queue