Combined RDB + AOF in Redis - Deep Dive

Overview - Combined RDB + AOF

What is it?

Combined RDB + AOF is a method Redis uses to save data safely by mixing two ways of saving: snapshots (RDB) and logs of every change (AOF). RDB saves the whole data at certain times, while AOF records every command that changes data. Using both together helps Redis recover data quickly and safely after a crash.

Why it matters

Without combining RDB and AOF, Redis might lose data or take too long to restart after a crash. RDB alone can lose recent changes, and AOF alone can be slow to load. Combining them solves these problems, making Redis reliable and fast, which is important for apps like chat, games, or shopping carts that need data saved safely and quickly.

Where it fits

Before learning this, you should understand basic Redis data storage and persistence methods like RDB snapshots and AOF logs separately. After this, you can explore Redis replication and high availability setups that build on persistence for fault tolerance.

Mental Model

Core Idea

Combining RDB snapshots and AOF logs lets Redis save data fast and recover it safely by using the strengths of both methods together.

Think of it like...

It's like saving your work by taking a full photo of your desk at the end of the day (RDB) and also writing down every change you make during the day in a notebook (AOF). If you lose your work, you can start from the photo and then replay the notebook to catch up.

┌─────────────┐       ┌───────────────┐
│  RDB Snapshot│──────▶│ Fast Restart  │
└─────────────┘       └───────────────┘
         │                    ▲
         │                    │
         ▼                    │
┌─────────────┐       ┌───────────────┐
│  AOF Log    │──────▶│ Data Recovery │
└─────────────┘       └───────────────┘
         ▲                    │
         └────────────────────┘
       Combined Persistence

Build-Up - 6 Steps

1

Foundation

Understanding RDB Snapshots

Concept: RDB saves the whole Redis data at once into a file at specific times.

Redis creates a snapshot of all data and saves it to disk as an RDB file. This happens every few minutes or when triggered. It is fast to load but can lose recent changes made after the last snapshot.

Result

You get a compact file representing the database state at a point in time.

Knowing RDB snapshots helps you understand how Redis can quickly restart but might lose recent data if it crashes.

2

Foundation

Understanding AOF Logs

Concept: AOF records every write command Redis receives to rebuild data step-by-step.

Instead of saving all data at once, AOF logs each command that changes data. When Redis restarts, it replays these commands to restore the database. This keeps data safer but can be slower to load.

Result

You get a log file that can replay all changes to recover the exact data state.

Understanding AOF shows how Redis can avoid data loss but may take longer to restart.

3

Intermediate

Why Combine RDB and AOF?

🤔Before reading on: do you think using only RDB or only AOF is enough for all Redis needs? Commit to your answer.

Concept: Combining RDB and AOF uses the speed of RDB and the safety of AOF together.

RDB is fast to load but can lose recent data. AOF is safer but slower to load. Redis can save an RDB snapshot and then append AOF commands after it. On restart, Redis loads the snapshot first, then replays the AOF commands after that snapshot point.

Result

Redis restarts faster than pure AOF and loses less data than pure RDB.

Knowing the tradeoffs between RDB and AOF explains why combining them improves both speed and safety.

4

Intermediate

How Redis Loads Combined Persistence

🤔Before reading on: do you think Redis loads AOF first or RDB first when both are used? Commit to your answer.

Concept: Redis loads the RDB snapshot first, then replays AOF commands that happened after the snapshot.

On restart, Redis reads the RDB file to quickly restore most data. Then it reads the AOF file from the point after the snapshot to apply recent changes. This two-step process speeds up recovery while keeping data safe.

Result

Redis recovers data quickly and with minimal loss.

Understanding the loading order clarifies how Redis balances speed and durability.

5

Advanced

Configuring Combined Persistence in Redis

🤔Before reading on: do you think Redis enables combined persistence by default or requires manual setup? Commit to your answer.

Concept: Redis allows configuring both RDB and AOF with settings to control how often snapshots and AOF fsync happen.

You can enable AOF with 'appendonly yes' and configure snapshot intervals with 'save' commands. Redis merges these settings so both persistence methods work together. You can also choose AOF fsync policies like always, every second, or no to balance safety and performance.

Result

Redis runs with combined persistence tuned to your needs.

Knowing configuration options helps you optimize Redis persistence for your application's needs.

6

Expert

Performance and Data Safety Tradeoffs in Combined Mode

🤔Before reading on: do you think combined persistence always guarantees zero data loss? Commit to your answer.

Concept: Combined persistence improves safety but still involves tradeoffs between speed and data loss risk depending on fsync settings.

If AOF fsync is set to every second, Redis may lose up to one second of data on crash. If set to always, performance slows down. RDB snapshots happen less often, so recent data is mostly recovered from AOF. Experts tune these settings based on application needs and hardware.

Result

You understand how to balance Redis persistence for your use case.

Knowing these tradeoffs prevents surprises in production and helps design reliable Redis systems.

Under the Hood

Redis writes RDB snapshots by forking the process and saving a compact binary dump of the dataset. AOF appends every write command to a log file. On restart, Redis loads the RDB file into memory, then reads and replays AOF commands after the snapshot point to restore the latest state. This layered approach reduces startup time compared to replaying the entire AOF from scratch.

Why designed this way?

Originally, Redis had separate RDB and AOF persistence to balance speed and durability. Combining them was designed to get the best of both worlds: fast restarts from snapshots and minimal data loss from command logs. This design avoids the slow startup of pure AOF and the data loss risk of pure RDB.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│  Redis Memory │──────▶│ Fork Process  │──────▶│ Write RDB File│
└───────────────┘       └───────────────┘       └───────────────┘
         │                      │                      │
         │                      ▼                      │
         │               ┌───────────────┐            │
         │               │ Append Commands│───────────┤
         │               │ to AOF Log    │            │
         │               └───────────────┘            │
         ▼                                            ▼
┌───────────────┐                               ┌───────────────┐
│  Restart Redis│◀──────────────────────────────│ Load RDB + AOF│
└───────────────┘                               └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does enabling AOF disable RDB snapshots automatically? Commit yes or no.

Common Belief: Many think that turning on AOF means Redis stops creating RDB snapshots.

Reality: Redis can run both RDB snapshots and AOF logging together; enabling AOF does not disable RDB by default.

Why it matters: If you assume RDB is off, you might miss the faster restart benefit and misconfigure persistence.

Quick: Does combined persistence guarantee zero data loss? Commit yes or no.

Common Belief: Some believe combined RDB + AOF means no data is ever lost on crash.

Reality: Data loss depends on AOF fsync settings; combined persistence reduces but does not eliminate data loss risk.

Why it matters: Overestimating durability can lead to data loss surprises in production.

Quick: Is loading AOF faster than loading RDB? Commit yes or no.

Common Belief: People often think AOF loads faster because it has all commands.

Reality: Loading RDB snapshots is much faster than replaying AOF commands, which is why Redis loads RDB first.

Why it matters: Misunderstanding load speed can cause wrong expectations about restart times.

Quick: Does Redis merge AOF and RDB files into one file? Commit yes or no.

Common Belief: Some think Redis combines AOF and RDB into a single file for persistence.

Reality: Redis keeps RDB and AOF as separate files and uses both during restart.

Why it matters: Confusing file formats can cause errors in backup and recovery procedures.

Expert Zone

1

The AOF rewrite process creates a new compact AOF file from the current dataset, reducing file size and speeding up recovery without blocking Redis.

2

Redis uses copy-on-write during RDB snapshotting to avoid blocking writes, but this can increase memory usage temporarily.

3

Tuning AOF fsync frequency impacts not only durability but also I/O load and latency, requiring careful balance in production.

When NOT to use

Combined persistence is not ideal when ultra-low latency is required and occasional data loss is acceptable; in such cases, disabling persistence or using only RDB snapshots may be better. For extremely high durability, external replication or disk-based databases might be preferred.

Production Patterns

In production, Redis often runs with combined persistence enabled, using AOF fsync every second and periodic RDB snapshots. Operators monitor AOF rewrite frequency and memory usage to avoid performance degradation. Backups include both RDB and AOF files for full recovery.

Connections

Write-Ahead Logging (WAL)

Combined AOF is similar to WAL in databases, where changes are logged before applying to data files.

Understanding WAL in databases helps grasp why Redis logs commands (AOF) to ensure durability alongside snapshots.

Checkpointing in Operating Systems

RDB snapshots act like checkpoints saving system state, while AOF logs are like event logs after checkpoints.

Knowing OS checkpointing clarifies how Redis balances fast recovery and data safety.

Version Control Systems

RDB snapshots are like commits capturing full project state, and AOF logs are like incremental changes between commits.

Seeing Redis persistence like version control helps understand incremental recovery and data consistency.

Common Pitfalls

#1 Assuming enabling AOF disables RDB snapshots automatically.

Wrong approach:

appendonly yes
# No save directives configured

Correct approach:

appendonly yes
save 900 1
save 300 10
save 60 10000

Root cause:

Misunderstanding Redis defaults and how persistence settings combine.

#2 Setting AOF fsync to 'no' for performance without realizing data loss risk.

Wrong approach:

appendfsync no

Correct approach:

appendfsync everysec

Root cause:

Prioritizing speed over durability without understanding fsync impact.

#3 Expecting Redis to load AOF faster than RDB snapshot.

Wrong approach:

Relying on AOF only and ignoring snapshot benefits.

Correct approach:

Use combined persistence to load RDB first, then AOF.

Root cause:

Not knowing the performance difference between loading snapshots and replaying logs.

Key Takeaways

Redis persistence combines RDB snapshots and AOF logs to balance fast recovery and data durability.

RDB snapshots save the full dataset periodically, enabling quick restarts but risking recent data loss.

AOF logs record every write command, ensuring safer data but slower recovery times.

Combined persistence loads the snapshot first, then replays AOF commands to restore the latest state efficiently.

Tuning AOF fsync and snapshot intervals is crucial to optimize Redis performance and data safety.