On-demand backups in DynamoDB - Deep Dive

Overview - On-demand backups

What is it?

On-demand backups are manual snapshots of your database taken whenever you choose. They capture the exact state of your data at that moment, allowing you to save it safely. You can restore your database to this saved state later if needed. This helps protect your data from accidental loss or corruption.

Why it matters

Without on-demand backups, you risk losing important data permanently if something goes wrong, like accidental deletion or software errors. These backups give you control to save your data at critical times, so you can recover quickly and avoid costly downtime or data loss. They are essential for protecting business information and maintaining trust.

Where it fits

Before learning on-demand backups, you should understand basic database concepts and how data is stored. After this, you can explore automated backups and disaster recovery strategies to build a full data protection plan.

Mental Model

Core Idea

An on-demand backup is like taking a photo of your database at a specific moment so you can return to that exact picture later if needed.

Think of it like...

Imagine you are writing a long essay by hand. Taking an on-demand backup is like taking a clear photo of your current page before continuing. If you make a mistake later, you can look at the photo and rewrite exactly what you had at that moment.

┌─────────────────────────────┐
│       Database Table         │
│  ┌───────────────┐          │
│  │ Current Data  │          │
│  └───────────────┘          │
│            │                │
│            ▼                │
│  ┌─────────────────────┐    │
│  │ On-demand Backup #1 │    │
│  └─────────────────────┘    │
│            │                │
│            ▼                │
│  ┌─────────────────────┐    │
│  │ On-demand Backup #2 │    │
│  └─────────────────────┘    │
└─────────────────────────────┘

Build-Up - 6 Steps

1

Foundation

What is an on-demand backup

Concept: Introducing the basic idea of manually saving a snapshot of your database.

An on-demand backup is a manual action where you tell the database to save all its current data exactly as it is. This snapshot is stored separately and can be used later to restore the database to this exact state. Unlike automatic backups, you decide when to create it.

Result

You get a saved copy of your database at a specific time that you control.

Understanding that on-demand backups are manual snapshots helps you see how you can protect data at important moments.

2

Foundation

How on-demand backups store data

Concept: Explaining what data is saved and how it is kept safe.

When you create an on-demand backup, the database copies all your table data and metadata into a secure storage. This backup is independent, so even if the original data changes or is deleted, the backup remains unchanged. It does not affect your live database performance.

Result

A complete, unchanging copy of your data is stored safely.

Knowing backups are independent copies clarifies why they are reliable for recovery.

3

Intermediate

Creating and managing backups in DynamoDB

🤔Before reading on: do you think creating an on-demand backup affects your database's availability? Commit to your answer.

Concept: How to create, list, and delete on-demand backups using DynamoDB tools.

In DynamoDB, you can create an on-demand backup using the AWS Console, CLI, or SDK. This process is fast and does not lock your table, so your app keeps working. You can list all backups to see their status and delete old backups to save storage costs.

Result

You can control when backups happen and manage their lifecycle without downtime.

Understanding that backups do not block your database helps you plan backups without disrupting users.

4

Intermediate

Restoring data from on-demand backups

🤔Before reading on: do you think restoring a backup overwrites your current database or creates a new one? Commit to your answer.

Concept: How to recover data by restoring from a backup and what options you have.

When you restore an on-demand backup in DynamoDB, you create a new table with the data from the backup. This means your current table stays unchanged until you switch to the restored one. This approach avoids accidental data loss and lets you test the restored data safely.

Result

You get a new table with the exact data from the backup time, ready to use or replace your current table.

Knowing restore creates a new table prevents accidental overwriting and data loss.

5

Advanced

Backup costs and performance considerations

🤔Before reading on: do you think on-demand backups slow down your database or increase costs significantly? Commit to your answer.

Concept: Understanding the impact of backups on cost and database performance.

On-demand backups in DynamoDB incur storage costs based on the size of your data. Creating backups does not slow down your database because it uses separate storage systems. However, frequent backups can increase your storage bills. Planning backup frequency balances data safety and cost.

Result

You can protect data without affecting performance but must manage storage costs.

Knowing cost and performance tradeoffs helps you design efficient backup strategies.

6

Expert

Consistency and backup timing surprises

🤔Before reading on: do you think an on-demand backup captures data exactly at the moment you request it, including all ongoing writes? Commit to your answer.

Concept: Exploring how DynamoDB handles data consistency during backups and what to expect.

DynamoDB on-demand backups are consistent and capture all data committed before the backup starts. However, writes happening exactly during backup initiation may or may not be included depending on timing. This means backups are consistent but not always a perfect instant snapshot of ongoing writes.

Result

Backups are reliable but understanding timing nuances prevents surprises in recovery.

Understanding backup consistency nuances helps avoid false assumptions about data state in recovery.

Under the Hood

DynamoDB on-demand backups create a point-in-time snapshot by copying data from the storage layer to a separate backup storage system. This process uses internal mechanisms to ensure data consistency without locking the table. The backup stores all table data and metadata, enabling full restoration later.

Why designed this way?

This design allows backups without downtime or performance impact, which is critical for always-on applications. Alternatives like locking tables during backup would cause service interruptions, so DynamoDB uses a snapshot approach to balance availability and data safety.

┌───────────────┐       ┌─────────────────────┐
│ DynamoDB Table│──────▶│ Backup Storage (S3) │
│  (Live Data)  │       │  (On-demand Backup)  │
└───────────────┘       └─────────────────────┘
       │                        ▲
       │                        │
       └───────────── Snapshot ─┘

Myth Busters - 4 Common Misconceptions

Quick: Does creating an on-demand backup pause your database writes? Commit yes or no.

Common Belief: Creating an on-demand backup stops all writes to keep data consistent.

Reality: DynamoDB backups do not pause writes; they use snapshot technology to capture consistent data without locking.

Why it matters: Believing backups pause writes may cause unnecessary downtime planning and fear of performance issues.

Quick: When restoring a backup, does it overwrite your existing table? Commit yes or no.

Common Belief: Restoring a backup replaces the current table automatically.

Reality: Restoring creates a new table; your current table remains unchanged until you switch over.

Why it matters: Assuming overwrite risks accidental data loss and confusion during recovery.

Quick: Do on-demand backups capture ongoing writes exactly at the backup request moment? Commit yes or no.

Common Belief: Backups capture every write instantly at the moment you request them.

Reality: Backups capture all committed data before backup starts; writes during backup may not be included.

Why it matters: Misunderstanding timing can lead to missing recent data after restore.

Quick: Are on-demand backups free of cost? Commit yes or no.

Common Belief: On-demand backups do not cost extra since they are part of the database service.

Reality: Backups incur storage costs based on data size and retention time.

Why it matters: Ignoring costs can lead to unexpected bills and budget issues.

Expert Zone

1

On-demand backups do not capture transient states of uncommitted transactions, ensuring only stable data is saved.

2

Backup storage is incremental under the hood, meaning only changes since the last backup consume additional space, optimizing cost.

3

Restoring from backup can be used for cloning tables to test changes without affecting production data.

When NOT to use

On-demand backups are not ideal for continuous data protection or point-in-time recovery; use continuous backups with point-in-time recovery (PITR) instead for those needs.

Production Patterns

Teams schedule on-demand backups before major updates or deployments to enable quick rollback. They also use backups to clone production data into test environments safely.

Connections

Point-in-time recovery (PITR)

Builds-on

Understanding on-demand backups helps grasp PITR, which automates continuous backups for more granular recovery.

Version control systems

Similar pattern

Both on-demand backups and version control snapshots save states at specific times, enabling rollback and comparison.

Photography

Metaphorical similarity

Taking a photo captures a moment in time, just like an on-demand backup captures your database state.

Common Pitfalls

#1 Assuming on-demand backups include uncommitted or in-progress writes.

Wrong approach:

Creating a backup during heavy write activity and expecting all recent changes to be included.

Correct approach:

Create backups during stable periods or understand that only committed data before backup start is included.

Root cause:

Misunderstanding how backup consistency works and timing of data capture.

#2 Restoring a backup directly over the existing table expecting automatic replacement.

Wrong approach:

Restoring backup with the same table name, expecting it to overwrite current data.

Correct approach:

Restore backup to a new table name, then switch application to use the restored table.

Root cause:

Not knowing that DynamoDB restores backups as new tables to prevent data loss.

#3 Ignoring storage costs and creating frequent unnecessary backups.

Wrong approach:

Scheduling on-demand backups every few minutes without cost consideration.

Correct approach:

Plan backup frequency based on data change rate and cost budget.

Root cause:

Lack of awareness about backup storage pricing and cost management.

Key Takeaways

On-demand backups let you manually save your database state at any time for safe recovery.

They create independent snapshots that do not affect your live database performance or availability.

Restoring a backup creates a new table, so your current data stays safe until you switch over.

Backups capture all committed data before the backup starts but may miss writes during backup initiation.

Planning backup frequency balances data safety with storage costs to avoid surprises.