Celery installation and setup in Django - Step-by-Step Execution

Concept Flow - Celery installation and setup

Install Celery package

↓

Create celery.py in Django project

↓

Configure Celery app with broker

↓

Update __init__.py to load Celery

↓

Run Celery worker

↓

Send tasks to Celery

↓

Tasks executed asynchronously

This flow shows the steps to install Celery, configure it in a Django project, and run tasks asynchronously.

Execution Sample

Django

pip install celery
# celery.py
from celery import Celery
app = Celery('proj', broker='redis://localhost:6379/0')

@app.task
def add(x, y):
    return x + y

This code installs Celery, creates a Celery app with Redis broker, and defines a simple task.

Execution Table

Step	Action	Code/Command	Result/Output
1	Install Celery package	pip install celery	Celery installed successfully
2	Create celery.py file	from celery import Celery app = Celery('proj', broker='redis://localhost:6379/0')	Celery app instance created with Redis broker
3	Define task function	@app.task def add(x, y): return x + y	Task 'add' registered with Celery
4	Update __init__.py	from .celery import app as celery_app __all__ = ('celery_app',)	Celery app loaded when Django starts
5	Run Celery worker	celery -A proj worker --loglevel=info	Worker starts and listens for tasks
6	Call task asynchronously	add.delay(4, 6)	Task sent to worker, returns AsyncResult
7	Worker executes task	add(4, 6)	Task runs, returns 10
8	Fetch result	result.get()	Returns 10
9	Exit	Stop worker	Worker stops, no more tasks processed

💡 Worker stopped, no more tasks processed

Variable Tracker

Variable	Start	After Step 3	After Step 6	After Step 7	Final
app	None	Celery instance created	Same instance	Same instance	Same instance
add	None	Task registered	Task called asynchronously	Task executed, result 10	Task available
result	None	None	AsyncResult object	Result value 10	Result value 10

Key Moments - 3 Insights

Why do we need to update __init__.py to import celery app?

Updating __init__.py ensures the Celery app is loaded when Django starts, so tasks are registered and workers can find them, as shown in step 4 of the execution_table.

What does add.delay(4, 6) do compared to add(4, 6)?

add.delay(4, 6) sends the task to the Celery worker to run asynchronously (step 6), while add(4, 6) runs the function immediately and synchronously (step 7).

Why do we need a message broker like Redis?

The broker (Redis) queues tasks and lets workers receive them asynchronously, as configured in step 2. Without it, Celery cannot distribute tasks.

Visual Quiz - 3 Questions

Look at the execution_table, what is the result of the task after step 7?

A 10

B 4

C 6

D None

💡 Hint

Check the 'Result/Output' column at step 7 where the task executes.

At which step is the Celery worker started to listen for tasks?

A Step 6

B Step 3

C Step 5

D Step 8

💡 Hint

Look for the step mentioning 'Run Celery worker' in the execution_table.

If we skip updating __init__.py, what will happen when Django starts?

A Celery app loads normally

B Tasks may not be registered and workers can't find them

C Redis broker will not start

D Celery worker will start automatically

💡 Hint

Refer to key_moments about __init__.py update and step 4 in execution_table.

Concept Snapshot

Celery installation and setup in Django:
1. Install Celery with pip.
2. Create celery.py with Celery app and broker config.
3. Define tasks with @app.task.
4. Import Celery app in __init__.py to load on Django start.
5. Run worker with 'celery -A proj worker'.
6. Call tasks asynchronously with task.delay().

Full Transcript

This visual execution shows how to install and set up Celery in a Django project. First, Celery is installed using pip. Then, a celery.py file is created where a Celery app instance is made and configured to use Redis as the message broker. Tasks are defined using the @app.task decorator. The __init__.py file is updated to import the Celery app so it loads when Django starts. Next, the Celery worker is run to listen for tasks. Tasks are sent asynchronously using the delay() method, and the worker executes them, returning results. The process stops when the worker is stopped. Variables like the Celery app instance, task function, and task result change state through these steps. Key points include why __init__.py must import the app, the difference between calling a task directly and asynchronously, and the role of the message broker. The quiz questions check understanding of task results, worker start step, and consequences of missing __init__.py update.

Practice

1. What is the main purpose of using Celery in a Django project?

easy

A. To replace Django's built-in ORM

B. To create database models automatically

C. To style the frontend of the Django app

D. To run time-consuming tasks in the background without blocking the main app

Solution

1. Step 1: Understand Celery's role

   Celery is a tool for running tasks asynchronously, meaning it handles tasks in the background.

2. Step 2: Compare options

   Creating database models automatically, styling the frontend of the Django app, and replacing Django's built-in ORM are unrelated to Celery; only running time-consuming tasks in the background without blocking the main app correctly describes its purpose.

3. Final Answer:

   To run time-consuming tasks in the background without blocking the main app -> Option D

4. Quick Check:

   Celery = background task runner [OK]

Hint: Celery runs slow tasks behind scenes to keep app fast [OK]

Common Mistakes:

• Thinking Celery manages database models
• Confusing Celery with frontend styling tools
• Assuming Celery replaces Django ORM

2. Which command correctly installs Celery and Redis as the message broker for a Django project?

easy

A. pip install celery redis

B. pip install django-celery

C. pip install celery-django redis-server

D. pip install celery-redis

Solution

1. Step 1: Identify the correct packages

   Celery requires the 'celery' package and a broker like Redis, installed via 'redis' package.

2. Step 2: Evaluate commands

   Only 'pip install celery redis' installs both needed packages correctly; others are incorrect or non-existent.

3. Final Answer:

   pip install celery redis -> Option A

4. Quick Check:

   Install celery and redis packages = pip install celery redis [OK]

Hint: Use pip install celery redis to get both packages [OK]

Common Mistakes:

• Installing 'django-celery' which is outdated
• Using non-existent package names
• Forgetting to install Redis client

3. Given this Celery task in Django:

@shared_task
def add(x, y):
    return x + y

result = add.delay(4, 5)
print(result.get(timeout=10))

What will be printed when this code runs correctly?

medium

A. 9

B. delay object

C. None

D. Error: get() not found

Solution

1. Step 1: Understand task execution

   The task 'add' runs asynchronously; calling 'delay(4, 5)' queues the task to add 4 and 5.

2. Step 2: Retrieve result with get()

   'result.get(timeout=10)' waits for the task to finish and returns the sum 9.

3. Final Answer:

   9 -> Option A

4. Quick Check:

   add(4,5) = 9 [OK]

Hint: delay queues task; get() fetches result [OK]

Common Mistakes:

• Expecting delay() to return result immediately
• Confusing task object with result value
• Not using get() to wait for task completion

4. You wrote this Celery setup in your Django project:

from celery import Celery
app = Celery('proj')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

But your tasks are not running. What is the most likely mistake?

medium

A. Using wrong Celery import statement

B. Not running a message broker like Redis or RabbitMQ

C. Missing @shared_task decorator on tasks

D. Not calling app.start() in settings.py

Solution

1. Step 1: Check broker setup

   Celery needs a running message broker (Redis or RabbitMQ) to send and receive tasks.

2. Step 2: Analyze other options

   Imports and decorators are correct; app.start() is not required in settings.py; missing broker is common cause.

3. Final Answer:

   Not running a message broker like Redis or RabbitMQ -> Option B

4. Quick Check:

   Broker must run for tasks to work [OK]

Hint: Always start Redis or RabbitMQ before Celery workers [OK]

Common Mistakes:

• Forgetting to start Redis or RabbitMQ server
• Assuming Celery works without a broker
• Misplacing @shared_task decorator

5. You want to configure Celery in your Django project to use Redis as the broker and ensure tasks are auto-discovered. Which of the following is the correct minimal setup in your celery.py file?

hard

A. from celery import Celery
app = Celery('proj')
app.config_from_object('settings')
app.autodiscover_tasks()

B. from celery import Celery
app = Celery('proj')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

C. from celery import Celery
app = Celery('proj', broker='redis://localhost:6379/0')
app.autodiscover_tasks()

D. from celery import Celery
app = Celery('proj')
app.broker_url = 'redis://localhost:6379/0'
app.autodiscover_tasks()

Solution

1. Step 1: Set broker URL directly in Celery instance

   from celery import Celery app = Celery('proj', broker='redis://localhost:6379/0') app.autodiscover_tasks() correctly passes Redis broker URL when creating Celery app, which is a minimal working setup.

2. Step 2: Evaluate other options

   The first uses an invalid path to settings ('settings' instead of 'django.conf:settings'); the second requires defining CELERY_BROKER_URL in Django settings.py first; the third is correct syntax-wise but setting broker_url attribute after creation is less minimal than passing in constructor.

3. Final Answer:

   from celery import Celery app = Celery('proj', broker='redis://localhost:6379/0') app.autodiscover_tasks() -> Option C

4. Quick Check:

   Broker URL in constructor = from celery import Celery app = Celery('proj', broker='redis://localhost:6379/0') app.autodiscover_tasks() [OK]

Hint: Pass broker URL directly to Celery constructor for quick setup [OK]

Common Mistakes:

• Setting broker_url attribute instead of passing in constructor
• Misconfiguring config_from_object with wrong settings path
• Omitting broker URL causing connection failure