Task retry and error handling in Django - Step-by-Step Execution

Concept Flow - Task retry and error handling

Start Task

↓

Try to execute task

↓

Task succeeds?

Yes→Finish Task

No↓

Check retry count < max retries?

Yes↓

Wait or schedule retry

↓

Retry task

Back to Try to execute task↓

Handle failure (log, alert)

↓

Finish Task with error

↩Back to Try to execute task

The task tries to run. If it fails, it checks if retries remain. If yes, it retries. If no, it handles the error and stops.

Execution Sample

Django

from celery import shared_task

@shared_task(bind=True, max_retries=3)
def send_email(self, email):
    try:
        # send email logic
        pass
    except Exception as exc:
        raise self.retry(exc=exc, countdown=5)

A Celery task tries to send an email, retries up to 3 times on failure with 5 seconds delay.

Execution Table

Attempt	Action	Exception Raised?	Retry Count	Next Step
1	Try sending email	Yes: ConnectionError	0	Schedule retry after 5s
2	Retry sending email	Yes: TimeoutError	1	Schedule retry after 5s
3	Retry sending email	Yes: SMTPServerError	2	Schedule retry after 5s
4	Retry sending email	Yes: SMTPServerError	3	Max retries reached, handle failure
End	Stop retries	N/A	3	Log error and alert

💡 Max retries (3) reached, task stops retrying and handles failure.

Variable Tracker

Variable	Start	After Attempt 1	After Attempt 2	After Attempt 3	After Attempt 4
retry_count	0	1	2	3	3
exception	None	ConnectionError	TimeoutError	SMTPServerError	SMTPServerError
task_state	Pending	Retrying	Retrying	Retrying	Failed

Key Moments - 3 Insights

Why does the task retry only 3 times and then stop?

Because max_retries is set to 3, so after the third retry (see execution_table row 4), the task stops retrying and handles the failure.

What happens if the task succeeds before max retries?

If the task succeeds, it finishes immediately without retrying (not shown in this trace but would skip retries).

Why do we use 'self.retry' inside the except block?

'self.retry' raises a retry exception that tells Celery to retry the task later, increasing retry_count and scheduling the next attempt.

Visual Quiz - 3 Questions

Look at the execution table, what is the retry_count after the second attempt?

A 0

B 2

C 1

D 3

💡 Hint

Check the 'Retry Count' column in row for Attempt 2 in execution_table.

At which attempt does the task stop retrying and handle failure?

A Attempt 4

B Attempt 2

C Attempt 3

D Attempt 5

💡 Hint

Look at the 'Next Step' column where it says 'Max retries reached' in execution_table.

If max_retries was set to 5, how would the execution_table change?

A The task would retry only 3 times as before.

B The task would retry 5 times before failure.

C The task would never retry.

D The task would retry only once.

💡 Hint

max_retries controls how many retries happen before stopping, see concept_flow and execution_table.

Concept Snapshot

Task retry and error handling in Django with Celery:
- Use @shared_task(bind=True, max_retries=N) to set retries.
- Wrap task code in try-except.
- On exception, call self.retry(exc=exc, countdown=seconds).
- Task retries up to max_retries times.
- After max retries, handle failure (log, alert).
- Helps recover from temporary errors automatically.

Full Transcript

This visual trace shows how a Django Celery task handles retries and errors. The task tries to send an email. If it fails, it raises an exception caught by the except block. The task then calls self.retry to schedule a retry after 5 seconds. The retry count increases each time. After 3 retries, the task stops retrying and handles the failure by logging or alerting. Variables like retry_count and exception change with each attempt. This pattern helps tasks recover from temporary problems by retrying automatically, improving reliability.