Periodic tasks with Celery Beat in Django - Step-by-Step Execution

Concept Flow - Periodic tasks with Celery Beat

Define Celery Task

↓

Configure Celery Beat Schedule

↓

Start Celery Worker

↓

Start Celery Beat Scheduler

↓

Scheduler triggers task at intervals

↓

Worker executes task

↓

Task completes and waits for next trigger

This flow shows how a periodic task is defined, scheduled, and executed repeatedly by Celery Beat and the Celery worker.

Execution Sample

Django

from celery import shared_task

@shared_task
def say_hello():
    print('Hello from Celery Beat!')

Defines a simple Celery task that prints a message when run.

Execution Table

Step	Action	Scheduler Time	Task Triggered	Worker Output
1	Celery Beat scheduler starts	00:00	No	No output
2	Scheduler checks schedule	00:05	No	No output
3	Scheduler time matches task interval	00:10	Yes	Task queued
4	Worker picks up task	00:10	Yes	Prints: Hello from Celery Beat!
5	Task completes	00:10	Yes	Idle until next trigger
6	Scheduler waits for next interval	00:15	No	No output
7	Scheduler time matches task interval again	00:20	Yes	Task queued
8	Worker executes task again	00:20	Yes	Prints: Hello from Celery Beat!
9	Task completes	00:20	Yes	Idle until next trigger

💡 The scheduler runs continuously; this trace shows two task executions at 10 and 20 seconds.

Variable Tracker

Variable	Start	After Step 3	After Step 4	After Step 7	After Step 8	Final
Scheduler Time	00:00	00:10	00:10	00:20	00:20	00:20
Task Triggered	No	Yes	Yes	Yes	Yes	Yes
Worker Output	No output	Task queued	Prints: Hello from Celery Beat!	Task queued	Prints: Hello from Celery Beat!	Idle until next trigger

Key Moments - 3 Insights

Why doesn't the task run immediately when the scheduler starts?

The scheduler waits until the first scheduled interval matches the current time before triggering the task, as shown in steps 1 and 2 where no task is triggered.

How does the worker know when to execute the task?

The scheduler queues the task at the scheduled time (step 3), and the worker continuously checks the queue to pick up and execute tasks (step 4).

Does the task run only once or repeatedly?

The task runs repeatedly at each scheduled interval, demonstrated by the repeated triggers and executions at steps 3-5 and 7-9.

Visual Quiz - 3 Questions

Look at the execution table, at which step does the worker first print the message?

A Step 4

B Step 3

C Step 2

D Step 1

💡 Hint

Check the 'Worker Output' column to see when the message is printed.

At which scheduler time does the task get triggered again after the first execution?

A 00:15

B 00:20

C 00:10

D 00:05

💡 Hint

Look at the 'Scheduler Time' and 'Task Triggered' columns for repeated triggers.

If the task interval was changed to 15 seconds, which step would change in the execution table?

A Step 3

B Step 7

C Step 6

D Step 9

💡 Hint

Changing the interval affects when the scheduler waits and triggers tasks, check the 'Scheduler waits' step.

Concept Snapshot

Periodic tasks with Celery Beat:
- Define tasks with @shared_task decorator.
- Configure schedule in Celery Beat settings.
- Run Celery worker and Beat scheduler.
- Beat triggers tasks at intervals.
- Worker executes tasks repeatedly.
- Useful for regular background jobs.

Full Transcript

This visual trace shows how periodic tasks work with Celery Beat in Django. First, you define a task using the @shared_task decorator. Then, you configure the schedule so Celery Beat knows when to run it. When you start the Celery worker and the Beat scheduler, the scheduler waits until the scheduled time to trigger the task. The worker picks up the task from the queue and runs it, printing the message. This process repeats at each interval, showing how Celery Beat automates periodic background jobs.

Practice

1. What is the main purpose of Celery Beat in a Django project?

easy

A. To schedule and run periodic tasks automatically

B. To handle HTTP requests asynchronously

C. To manage database migrations

D. To serve static files efficiently

Solution

1. Step 1: Understand Celery Beat's role

   Celery Beat is designed to schedule tasks to run at specific times or intervals automatically.

2. Step 2: Differentiate from other components

   Handling HTTP requests, managing migrations, or serving static files are not functions of Celery Beat.

3. Final Answer:

   To schedule and run periodic tasks automatically -> Option A

4. Quick Check:

   Celery Beat = periodic task scheduler [OK]

Hint: Celery Beat schedules tasks, not handles requests [OK]

Common Mistakes:

• Confusing Celery Beat with Django's request handling
• Thinking Celery Beat manages database or static files
• Assuming Celery Beat runs tasks immediately without schedule

2. Which of the following is the correct way to define a periodic task schedule using Celery Beat with a crontab in Django settings?

easy

A. "beat_schedule = { 'task-name': { 'task': 'app.tasks.my_task', 'schedule': crontab(minute='0', hour='*/3') } }"

B. "beat_schedule = { 'task-name': { 'task': 'app.tasks.my_task', 'schedule': crontab('0', '*/3') } }"

C. "beat_schedule = { 'task-name': { 'task': 'app.tasks.my_task', 'schedule': crontab(minute=0, hour='every 3 hours') } }"

D. "beat_schedule = { 'task-name': { 'task': 'app.tasks.my_task', 'schedule': crontab(minute='0', hour='3/0') } }"

Solution

1. Step 1: Check crontab syntax

   The crontab function requires named arguments like minute and hour with string values representing schedule patterns.

2. Step 2: Validate correct usage

   "beat_schedule = { 'task-name': { 'task': 'app.tasks.my_task', 'schedule': crontab(minute='0', hour='*/3') } }" correctly uses crontab(minute='0', hour='*/3') to run every 3 hours at minute 0.

3. Final Answer:

   beat_schedule with crontab(minute='0', hour='*/3') -> Option A

4. Quick Check:

   Correct crontab syntax = "beat_schedule = { 'task-name': { 'task': 'app.tasks.my_task', 'schedule': crontab(minute='0', hour='*/3') } }" [OK]

Hint: Use named args with strings in crontab() [OK]

Common Mistakes:

• Passing positional arguments instead of named
• Using invalid hour format like '3/0'
• Mixing string and integer types incorrectly

3. Given this Celery Beat schedule snippet in Django settings:

beat_schedule = {
  'print-time': {
    'task': 'app.tasks.print_time',
    'schedule': crontab(minute='*/15')
  }
}

What will happen when Celery Beat and worker run?

medium

A. The task will not run due to syntax error

B. The task runs only once at minute 15

C. The task runs every hour at minute 0

D. The task 'print_time' runs every 15 minutes

Solution

1. Step 1: Interpret crontab(minute='*/15')

   This means the task runs every 15 minutes, at minute 0, 15, 30, 45 of each hour.

2. Step 2: Understand Celery Beat behavior

   When both Beat and worker run, Beat triggers the task on schedule, so it runs repeatedly every 15 minutes.

3. Final Answer:

   The task 'print_time' runs every 15 minutes -> Option D

4. Quick Check:

   crontab '*/15' means every 15 minutes [OK]

Hint: */15 in crontab means every 15 minutes [OK]

Common Mistakes:

• Thinking it runs only once
• Confusing minute '*/15' with fixed minute 15
• Assuming syntax error without checking carefully

4. You defined this schedule in your Django settings:

beat_schedule = {
  'cleanup-task': {
    'task': 'app.tasks.cleanup',
    'schedule': crontab(minute=0, hour='*')
  }
}

But the task never runs. What is the most likely cause?

medium

A. The task name 'cleanup-task' is invalid

B. You forgot to start the Celery Beat service

C. The crontab syntax is incorrect because minute should be a string

D. The worker must be restarted every hour

Solution

1. Step 1: Check if Celery Beat is running

   Celery Beat must be running to send scheduled tasks to the worker.

2. Step 2: Validate other options

   Task names can be any string, crontab accepts integer or string for minute, and workers do not need hourly restart.

3. Final Answer:

   You forgot to start the Celery Beat service -> Option B

4. Quick Check:

   Beat service must run for schedules to trigger [OK]

Hint: Always run Celery Beat alongside worker [OK]

Common Mistakes:

• Assuming task name format causes failure
• Thinking crontab minute must be string only
• Restarting worker unnecessarily

5. You want to run a Django Celery task every day at 2:30 AM and also every 10 minutes. How should you configure beat_schedule to achieve this?

hard

A. { 'daily-task': { 'task': 'app.tasks.daily', 'schedule': crontab(hour='2:30') }, 'frequent-task': { 'task': 'app.tasks.frequent', 'schedule': crontab(minute=10) } }

B. { 'daily-task': { 'task': 'app.tasks.daily', 'schedule': crontab(hour=2, minute=30) }, 'frequent-task': { 'task': 'app.tasks.frequent', 'schedule': crontab(minute=10) } }

C. { 'daily-task': { 'task': 'app.tasks.daily', 'schedule': crontab(hour=2, minute=30) }, 'frequent-task': { 'task': 'app.tasks.frequent', 'schedule': crontab(minute='*/10') } }

D. { 'daily-task': { 'task': 'app.tasks.daily', 'schedule': crontab(hour='2', minute='30') }, 'frequent-task': { 'task': 'app.tasks.frequent', 'schedule': crontab(minute='10') } }

Solution

1. Step 1: Define daily task schedule correctly

   Use crontab(hour=2, minute=30) or with strings '2' and '30' to run at 2:30 AM daily.

2. Step 2: Define frequent task schedule correctly

   Use crontab(minute='*/10') to run every 10 minutes; minute=10 runs only at minute 10 each hour.

3. Step 3: Check all options

   { 'daily-task': { 'task': 'app.tasks.daily', 'schedule': crontab(hour=2, minute=30) }, 'frequent-task': { 'task': 'app.tasks.frequent', 'schedule': crontab(minute='*/10') } } uses correct crontab syntax for both tasks; others have invalid formats or misunderstandings.

4. Final Answer:

   Use crontab(hour=2, minute=30) and crontab(minute='*/10') -> Option C

5. Quick Check:

   Daily at 2:30 and every 10 min = { 'daily-task': { 'task': 'app.tasks.daily', 'schedule': crontab(hour=2, minute=30) }, 'frequent-task': { 'task': 'app.tasks.frequent', 'schedule': crontab(minute='*/10') } } [OK]

Hint: Use '*/10' for every 10 minutes, not minute=10 [OK]

Common Mistakes:

• Using '2:30' as hour value
• Setting minute=10 instead of '*/10' for intervals
• Mixing string and integer types incorrectly