Periodic tasks with Celery Beat in Flask - Step-by-Step Execution

Concept Flow - Periodic tasks with Celery Beat

Define Celery app

↓

Configure Celery Beat schedule

↓

Start Celery worker and Beat

↓

Beat triggers task periodically

↓

Worker executes task

↓

Task completes and waits for next trigger

This flow shows how Celery Beat schedules tasks periodically and the worker executes them repeatedly.

Execution Sample

Flask

from celery import Celery
app = Celery('tasks', broker='redis://localhost')
app.conf.beat_schedule = {
  'say-hello-every-10-seconds': {
    'task': 'tasks.say_hello', 'schedule': 10.0
  }
}
@app.task
def say_hello():
  print('Hello!')

This code sets up a Celery app with a Beat schedule to run say_hello every 10 seconds.

Execution Table

Step	Action	Beat Schedule Check	Task Triggered	Worker Executes	Output
1	Start Celery Beat and Worker	Beat running, waiting 10s	No	No
2	10 seconds passed	Beat triggers say_hello	Yes	Yes	Hello!
3	Task completes	Beat resets timer	No	No
4	Another 10 seconds passed	Beat triggers say_hello	Yes	Yes	Hello!
5	Task completes	Beat resets timer	No	No

💡 This process repeats indefinitely until stopped.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	Final
Beat Timer	0s	10s	0s	10s	0s	Repeats
Task Triggered	False	True	False	True	False	Repeats
Worker State	Idle	Running say_hello	Idle	Running say_hello	Idle	Repeats

Key Moments - 2 Insights

Why doesn't the task run immediately when Celery Beat starts?

Celery Beat waits for the first schedule interval (10 seconds) before triggering the task, as shown in execution_table step 1 where the timer counts up before triggering.

How does Celery Beat know when to trigger the task again?

After each task execution, Beat resets its timer and waits for the next interval, visible in execution_table steps 3 and 5 where the timer resets to 0s.

Visual Quiz - 3 Questions

Look at the execution_table at step 2, what is the worker doing?

A Executing the say_hello task

B Idle, waiting for task

C Starting Celery Beat

D Stopping the worker

💡 Hint

Check the 'Worker Executes' column at step 2 in execution_table

At which step does the Beat reset its timer after task completion?

A Step 1

B Step 4

C Step 3

D Step 5

💡 Hint

Look at the 'Beat Schedule Check' column for timer reset in execution_table

If the schedule was changed to 5 seconds, how would the 'Beat Timer' variable change in variable_tracker?

A It would count to 10 seconds before resetting

B It would count to 5 seconds before resetting

C It would never reset

D It would reset randomly

💡 Hint

Refer to the 'Beat Timer' row in variable_tracker and relate to schedule timing

Concept Snapshot

Periodic tasks with Celery Beat:
- Define Celery app and tasks
- Configure beat_schedule with task and interval
- Start Celery worker and Beat together
- Beat triggers tasks on schedule
- Worker executes tasks repeatedly
- Process runs continuously until stopped

Full Transcript

This visual execution shows how Celery Beat works with a Flask Celery app to run periodic tasks. First, the Celery app is defined with a Redis broker. Then, the beat_schedule config sets a task to run every 10 seconds. When Celery Beat and the worker start, Beat waits 10 seconds before triggering the task. The worker runs the task, printing 'Hello!'. After the task finishes, Beat resets its timer and waits for the next 10 seconds. This cycle repeats indefinitely. Variables like the Beat timer and worker state change accordingly. Key moments include understanding why the task doesn't run immediately and how Beat resets its timer. The quizzes test understanding of worker actions, timer resets, and schedule changes.

Practice

1. What is the main purpose of Celery Beat in a Flask application?

easy

A. To schedule tasks to run automatically at specific times

B. To handle HTTP requests asynchronously

C. To manage database migrations

D. To serve static files efficiently

Solution

1. Step 1: Understand Celery Beat's role

   Celery Beat is a scheduler that triggers tasks periodically without manual intervention.

2. Step 2: Compare with other options

   Handling HTTP requests, managing migrations, and serving static files are unrelated to scheduling tasks.

3. Final Answer:

   To schedule tasks to run automatically at specific times -> Option A

4. Quick Check:

   Celery Beat = Task scheduler [OK]

Hint: Celery Beat is all about timing tasks automatically [OK]

Common Mistakes:

• Confusing Celery Beat with request handling
• Thinking it manages database or static files
• Assuming it runs tasks only once

2. Which of the following is the correct way to define a periodic task in Celery Beat using Flask?

easy

A. @app.route('/task')\ndef my_task():\n print('Hello')

B. @celery.task\ndef my_task():\n print('Hello')

C. def my_task():\n print('Hello')

D. @celery.task\nasync def my_task():\n print('Hello')

Solution

1. Step 1: Identify task decorator

   Periodic tasks in Celery are defined with the @celery.task decorator.

2. Step 2: Check syntax correctness

   @celery.task\ndef my_task():\n print('Hello') correctly uses @celery.task and a normal function. @app.route('/task')\ndef my_task():\n print('Hello') uses Flask route decorator, which is unrelated. def my_task():\n print('Hello') lacks any decorator. @celery.task\nasync def my_task():\n print('Hello') uses async which is not standard for Celery tasks.

3. Final Answer:

   @celery.task\ndef my_task():\n print('Hello') -> Option B

4. Quick Check:

   Use @celery.task to define Celery tasks [OK]

Hint: Use @celery.task decorator for Celery tasks [OK]

Common Mistakes:

• Using Flask route decorator instead of @celery.task
• Defining tasks without any decorator
• Using async def which Celery does not support by default

3. Given this Celery Beat schedule snippet, what will happen?

beat_schedule = {
    'print-every-10-seconds': {
        'task': 'app.print_message',
        'schedule': 10,
    },
}

medium

A. The task runs once after 10 seconds

B. The task runs every 10 minutes

C. The task 'print_message' runs every 10 seconds

D. This code causes a syntax error

Solution

1. Step 1: Understand the schedule value

   The 'schedule' key with value 10 means the task runs every 10 seconds.

2. Step 2: Confirm task repetition

   Celery Beat schedules tasks repeatedly at the interval given, so it runs every 10 seconds, not once or every 10 minutes.

3. Final Answer:

   The task 'print_message' runs every 10 seconds -> Option C

4. Quick Check:

   Schedule=10 means every 10 seconds [OK]

Hint: Schedule number means seconds interval by default [OK]

Common Mistakes:

• Thinking schedule is minutes instead of seconds
• Assuming task runs only once
• Confusing syntax with crontab format

4. Identify the error in this Celery Beat configuration:

beat_schedule = {
    'task1': {
        'task': 'app.task1',
        'schedule': crontab(minute='*/5')
    },
    'task2': {
        'task': 'app.task2',
        'schedule': '10',
    }
}

medium

A. The dictionary keys 'task1' and 'task2' must be integers

B. 'task' names must be functions, not strings

C. Missing import for crontab causes runtime error

D. 'schedule' for 'task2' should be an integer or crontab, not a string

Solution

1. Step 1: Check 'schedule' types

   For 'task1', schedule uses crontab correctly. For 'task2', schedule is a string '10', which should be an integer 10 or a crontab object.

2. Step 2: Validate other parts

   Task names as strings are correct references. Import errors are not shown here. Dictionary keys can be strings.

3. Final Answer:

   'schedule' for 'task2' should be an integer or crontab, not a string -> Option D

4. Quick Check:

   Schedule must be int or crontab, not string [OK]

Hint: Schedule values must be int or crontab, not quoted strings [OK]

Common Mistakes:

• Using string instead of int for schedule interval
• Confusing task names with function objects
• Assuming dictionary keys must be integers

5. You want to run a Flask Celery task every Monday at 8:30 AM using Celery Beat. Which beat_schedule entry is correct?

hard

A. {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week='mon')}

B. {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week=2)}

C. {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week='tue')}

D. {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week='Monday')}

Solution

1. Step 1: Understand crontab day_of_week format

   Celery's crontab accepts day_of_week as string names like 'mon', 'tue', or integers 0-6 (0=Sunday).

2. Step 2: Evaluate options

   {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week='mon')} uses 'mon' string which is correct. {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week=2)} uses integer 2 which is Tuesday (0=Sunday). {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week='tue')} uses 'tue' string which is Tuesday. {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week='Monday')} uses 'Monday' full name which is not supported.

3. Final Answer:

   {'task': 'app.weekly_task', 'schedule': crontab(hour=8, minute=30, day_of_week='mon')} -> Option A

4. Quick Check:

   Use 'mon' string for Monday in crontab [OK]

Hint: Use three-letter lowercase day names like 'mon' for crontab [OK]

Common Mistakes:

• Using full day names like 'Monday' instead of 'mon'
• Using string numbers like '1' instead of integers or names
• Confusing day_of_week numbering starting at Sunday