Background file processing in FastAPI

Introduction

Background file processing lets your app handle big or slow tasks without making users wait. It runs tasks behind the scenes while your app stays fast and responsive.

Uploading large files that need extra work like resizing or scanning

Processing files after a user submits a form without blocking the response

Running virus scans or validations on uploaded files without delay

Generating reports or thumbnails from uploaded files after saving

Any time you want to keep the app quick while doing slow file tasks

Syntax

FastAPI

from fastapi import FastAPI, BackgroundTasks, UploadFile

app = FastAPI()

def process_file(file_path: str):
    # Your file processing logic here
    pass

@app.post("/upload")
async def upload_file(background_tasks: BackgroundTasks, file: UploadFile):
    file_location = f"files/{file.filename}"
    with open(file_location, "wb") as f:
        f.write(await file.read())
    background_tasks.add_task(process_file, file_location)
    return {"message": "File uploaded and processing started"}

Use BackgroundTasks to add tasks that run after the response is sent.

File processing functions should be normal functions, not async, to avoid blocking.

Examples

A simple background task that prints the file path.

FastAPI

def process_file(file_path: str):
    print(f"Processing {file_path}")

Adds the process_file function to run in the background with the file path argument.

FastAPI

background_tasks.add_task(process_file, "files/myfile.txt")

Uploads a file, saves it, and starts background processing without waiting.

FastAPI

@app.post("/upload")
async def upload(background_tasks: BackgroundTasks, file: UploadFile):
    file_location = f"files/{file.filename}"
    with open(file_location, "wb") as f:
        f.write(await file.read())
    background_tasks.add_task(process_file, file_location)
    return {"message": "Upload successful"}

Sample Program

This FastAPI app uploads a file, saves it, and then processes it in the background. The user gets an immediate response while the file is processed behind the scenes.

FastAPI

from fastapi import FastAPI, BackgroundTasks, UploadFile
import time

app = FastAPI()

def process_file(file_path: str):
    # Simulate a slow file processing task
    print(f"Start processing {file_path}")
    time.sleep(3)  # pretend to do work
    print(f"Finished processing {file_path}")

@app.post("/upload")
async def upload(background_tasks: BackgroundTasks, file: UploadFile):
    file_location = f"files/{file.filename}"
    with open(file_location, "wb") as f:
        f.write(await file.read())
    background_tasks.add_task(process_file, file_location)
    return {"message": "File uploaded and processing started"}

Important Notes

Background tasks run after the response is sent, so users don't wait.

Make sure the file saving happens before starting background processing.

Background tasks are good for small to medium jobs; for heavy jobs consider task queues like Celery.

Summary

Background file processing keeps your app fast by running slow tasks after response.

Use FastAPI's BackgroundTasks to add these tasks easily.

Always save files first, then start background processing.