Why databases persist data in FastAPI - Why It Works This Way

Overview - Why databases persist data

What is it?

Databases are systems that store data so it can be saved and used later. Persisting data means keeping it safe even when the computer or app is turned off. This allows apps to remember information like user accounts, messages, or settings over time. Without persistence, all data would disappear when the app closes.

Why it matters

Persisting data solves the problem of losing important information every time an app stops running. Imagine writing a message and losing it when your phone restarts. Databases keep data safe and organized so apps can work reliably and users don’t lose their work or preferences. Without this, apps would be forgetful and frustrating.

Where it fits

Before learning why databases persist data, you should understand basic programming and how apps handle data temporarily. After this, you can learn about different types of databases, how to connect them to apps like FastAPI, and how to query and update data safely.

Mental Model

Core Idea

Databases persist data by saving it permanently on storage so it remains available even after the app or computer stops running.

Think of it like...

Persisting data in a database is like writing notes in a notebook instead of on a whiteboard. The notebook keeps your notes safe over time, while the whiteboard erases everything when you clean it.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│   App Memory  │──────▶│   Database    │──────▶│  Storage Disk │
│ (Temporary)   │       │ (Persistent)  │       │ (Permanent)   │
└───────────────┘       └───────────────┘       └───────────────┘

Build-Up - 6 Steps

1

Foundation

What is data persistence

Concept: Data persistence means saving data so it lasts beyond the current program run.

When you use an app, it stores some data temporarily in memory. But when the app closes, that data disappears. Persistence means saving data to a place that keeps it safe even when the app is off, like a file or database.

Result

You understand that persistence is about saving data permanently, not just temporarily.

Understanding persistence is key to knowing why databases exist and how they help apps remember information.

2

Foundation

Difference between memory and storage

Concept: Memory is fast but temporary; storage is slower but permanent.

Computers have memory (RAM) that apps use to work quickly. But RAM forgets everything when power is lost. Storage devices like hard drives or SSDs keep data safe even when the computer is off.

Result

You can explain why data in memory disappears but data in storage stays.

Knowing this difference helps you see why databases use storage to persist data.

3

Intermediate

How databases save data permanently

🤔Before reading on: do you think databases save data directly in memory or on disk? Commit to your answer.

Concept: Databases write data to storage disks to keep it safe and permanent.

When you save data in a database, it writes that data to files on a disk. This way, even if the app or computer restarts, the data is still there. Databases also organize data so it can be found and updated efficiently.

Result

You understand that databases use storage devices to persist data beyond app runs.

Knowing that databases rely on disk storage explains how they keep data safe long-term.

4

Intermediate

Why apps use databases for persistence

🤔Before reading on: do you think apps can just save data in files instead of databases? Commit to your answer.

Concept: Databases provide organized, reliable, and fast ways to save and retrieve data compared to simple files.

While apps can save data in files, databases offer features like searching, updating, and protecting data from errors. They handle many users and large data efficiently, which simple files cannot do well.

Result

You see why databases are preferred for complex or shared data persistence.

Understanding the advantages of databases over files helps you appreciate their role in app development.

5

Advanced

How databases ensure data safety

🤔Before reading on: do you think databases always save data instantly or sometimes delay it? Commit to your answer.

Concept: Databases use techniques like transactions and logging to keep data safe and consistent even if errors or crashes happen.

Databases group changes into transactions that either fully happen or not at all. They also keep logs to recover data if the system crashes. This prevents data loss or corruption.

Result

You understand how databases protect data integrity during saves.

Knowing these safety mechanisms explains why databases are trusted for critical data.

6

Expert

Trade-offs in database persistence design

🤔Before reading on: do you think faster data saving always means safer data? Commit to your answer.

Concept: Database persistence balances speed, safety, and complexity with design choices like caching and write strategies.

To save data quickly, databases may delay writing to disk or use memory caches. But this risks losing recent changes if a crash occurs. Designers choose trade-offs based on app needs, balancing performance and data safety.

Result

You see why database persistence is a complex design problem with no perfect solution.

Understanding these trade-offs helps you make informed choices when building or using databases.

Under the Hood

Databases persist data by writing structured data to storage media like hard drives or SSDs. They use files and indexes to organize data for fast access. When data changes, databases write logs and use transactions to ensure all changes complete fully or not at all. This prevents partial updates and corruption. The database engine manages caching data in memory for speed but flushes changes to disk to keep data permanent.

Why designed this way?

Databases were designed to solve the problem of safely storing and retrieving large amounts of data reliably. Early systems used simple files but faced issues with data loss, corruption, and slow access. The transactional model and logging were introduced to guarantee data integrity. Balancing speed and safety led to caching and delayed writes. These design choices evolved to meet the needs of multi-user, high-availability systems.

┌───────────────┐
│   Application │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Database Engine│
│ ┌───────────┐ │
│ │ Transaction│ │
│ │  Manager   │ │
│ └────┬──────┘ │
│      │        │
│ ┌────▼──────┐ │
│ │  Cache    │ │
│ └────┬──────┘ │
└──────┼────────┘
       │ Writes
       ▼
┌───────────────┐
│ Storage Disk  │
│ (Files & Logs)│
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do databases always save data immediately to disk? Commit to yes or no.

Common Belief: Databases write data instantly to disk every time you save.

Reality: Databases often cache data in memory and write to disk later to improve speed.

Why it matters: Assuming instant writes can lead to misunderstanding data loss risks during crashes.

Quick: Can you use any file as a database? Commit to yes or no.

Common Belief: Any file can act as a database if you save data there.

Reality: Databases use special structures and indexes to organize data efficiently, unlike simple files.

Why it matters: Treating files as databases can cause slow, unreliable data access in real apps.

Quick: Does persisting data mean it never changes? Commit to yes or no.

Common Belief: Persisted data is fixed and cannot be updated.

Reality: Databases allow data to be updated, deleted, or added while keeping it persistent.

Why it matters: Thinking persistence means unchangeable data limits understanding of dynamic apps.

Quick: Is data persistence only about saving data? Commit to yes or no.

Common Belief: Persistence just means saving data somewhere.

Reality: Persistence also involves organizing, protecting, and efficiently accessing data over time.

Why it matters: Ignoring these aspects leads to poor app performance and data errors.

Expert Zone

1

Some databases use write-ahead logging to ensure data can be recovered after crashes without losing committed changes.

2

The choice between synchronous and asynchronous disk writes affects both performance and risk of data loss.

3

Database caching strategies can cause data to appear saved but not yet be on disk, requiring careful transaction management.

When NOT to use

Databases are not ideal for very simple or temporary data storage where file systems or in-memory caches suffice. For high-speed, ephemeral data, in-memory stores like Redis are better. For unstructured large files, object storage is preferred.

Production Patterns

In real systems, databases are combined with caching layers, backups, and replication to ensure data persistence, availability, and performance. Developers use ORMs or query builders in FastAPI to interact with databases safely and efficiently.

Connections

File Systems

Databases build on file systems by adding structure and management for data persistence.

Understanding file systems helps grasp how databases store data physically and why they need extra layers.

Memory Management

Databases use memory management techniques like caching to balance speed and persistence.

Knowing memory management clarifies why databases delay writes and how they optimize performance.

Human Memory and Note Taking

Persistence in databases is like how humans write notes to remember information beyond short-term memory.

This connection shows how persistence solves the universal problem of retaining information over time.

Common Pitfalls

#1 Assuming data is saved permanently immediately after a save command.

Wrong approach:

await db.save(data)
// Assume data is safely on disk now

Correct approach:

await db.transaction(async () => {
await db.save(data)
})
// Ensures data is committed safely

Root cause:

Misunderstanding that database writes may be cached and not instantly persisted.

#2 Using plain files to store complex app data without structure.

Wrong approach:

fs.writeFile('data.txt', JSON.stringify(data))
// No indexing or query support

Correct approach:

Use a database like SQLite or PostgreSQL to store and query data efficiently.

Root cause:

Not recognizing the need for organized data storage and retrieval.

#3 Ignoring transaction management leading to partial data updates.

Wrong approach:

db.updatePart1()
db.updatePart2()
// No transaction wrapping

Correct approach:

db.transaction(async () => {
await db.updatePart1()
await db.updatePart2()
})

Root cause:

Not understanding atomicity and consistency in data persistence.

Key Takeaways

Databases persist data by saving it permanently on storage devices, allowing apps to remember information beyond their runtime.

Persistence solves the problem of data loss when apps or computers shut down, making apps reliable and user-friendly.

Databases organize, protect, and efficiently manage data, which simple files cannot do well for complex or shared data.

Database persistence involves trade-offs between speed and safety, using techniques like caching, transactions, and logging.

Understanding how databases persist data helps developers build better apps and avoid common mistakes with data loss or corruption.

Practice

1. Why do databases persist data in applications like FastAPI?

easy

A. To keep data safe even if the app stops or restarts

B. To make the app run faster

C. To delete old data automatically

D. To prevent users from accessing the app

Solution

1. Step 1: Understand what persistence means

   Persistence means data stays saved even after the program stops running.

2. Step 2: Connect persistence to databases in FastAPI

   Databases store data on disk, so FastAPI can retrieve it later, even after restarts.

3. Final Answer:

   To keep data safe even if the app stops or restarts -> Option A

4. Quick Check:

   Persistence means data stays saved [OK]

Hint: Persistence means data stays saved after app stops [OK]

Common Mistakes:

• Thinking databases speed up the app only
• Confusing persistence with data deletion
• Believing databases block user access

2. Which of the following is the correct way to save data to a database in FastAPI?

easy

A. Use a database session to add and commit the data

B. Print the data to the console

C. Store data in a local variable only

D. Use a global variable to hold data

Solution

1. Step 1: Identify how FastAPI interacts with databases

   FastAPI uses database sessions to add and commit data to save it permanently.

2. Step 2: Compare options for saving data

   Printing or using variables does not save data persistently; only committing via session does.

3. Final Answer:

   Use a database session to add and commit the data -> Option A

4. Quick Check:

   Commit data with session to save [OK]

Hint: Commit data with session to save persistently [OK]

Common Mistakes:

• Thinking printing saves data
• Using variables instead of database commit
• Skipping the commit step

3. Given this FastAPI code snippet, what will happen when the app restarts?

from fastapi import FastAPI
from sqlalchemy.orm import Session

app = FastAPI()

@app.post('/items/')
def create_item(session: Session, item: Item):
    session.add(item)
    session.commit()
    return item

medium

A. The items will be saved only in memory

B. The saved items will be lost after restart

C. The app will crash on restart

D. The saved items will still be available after restart

Solution

1. Step 1: Understand what session.commit() does

   Calling commit() saves data permanently to the database storage.

2. Step 2: Consider app restart effect on database data

   Since data is saved in the database, it remains after the app restarts.

3. Final Answer:

   The saved items will still be available after restart -> Option D

4. Quick Check:

   Committed data persists after restart [OK]

Hint: Committed data stays after app restarts [OK]

Common Mistakes:

• Confusing commit with temporary memory storage
• Assuming app restart clears database
• Thinking session.add alone saves data

4. What is wrong with this FastAPI code that tries to save data?

from fastapi import FastAPI

app = FastAPI()

@app.post('/users/')
def create_user(user: dict):
    user['id'] = 1
    return user

medium

A. It uses the wrong HTTP method

B. It does not save data to a database, so data is lost on restart

C. It has a syntax error in the function

D. It commits data twice

Solution

1. Step 1: Check if data is saved to a database

   The code only modifies and returns a dictionary; it does not save to any database.

2. Step 2: Understand consequence of no database saving

   Without saving to a database, data is lost when the app stops or restarts.

3. Final Answer:

   It does not save data to a database, so data is lost on restart -> Option B

4. Quick Check:

   No database save means no persistence [OK]

Hint: Data must be saved to database for persistence [OK]

Common Mistakes:

• Thinking returning dict saves data
• Confusing HTTP method with persistence
• Assuming syntax error when none exists

5. You want to ensure user data is saved permanently in FastAPI. Which approach correctly combines FastAPI and database persistence?

hard

A. Store user data in a global list variable inside the app

B. Save user data only in request body without database interaction

C. Use FastAPI with SQLAlchemy session to add and commit user data to a database

D. Print user data to console and rely on logs for storage

Solution

1. Step 1: Identify how to save data permanently

   Permanent saving requires writing data to a database, not just memory or logs.

2. Step 2: Match FastAPI with database usage

   FastAPI works well with SQLAlchemy sessions to add and commit data to databases.

3. Step 3: Eliminate incorrect options

   Global variables and printing do not persist data after app stops; request body alone is temporary.

4. Final Answer:

   Use FastAPI with SQLAlchemy session to add and commit user data to a database -> Option C

5. Quick Check:

   Database commit ensures permanent data [OK]

Hint: Use database session commit for permanent save [OK]

Common Mistakes:

• Using global variables for persistence
• Relying on request data without saving
• Thinking console logs save data