Memory and storage engine basics (WiredTiger) in MongoDB - Step-by-Step Execution

Concept Flow - Memory and storage engine basics (WiredTiger)

Start MongoDB Instance

↓

Initialize WiredTiger Engine

↓

Allocate Memory Cache

↓

Manage Data in Cache

↓

Write Data to Disk

↓

Read Data from Disk

↓

Serve Queries Using Cache and Disk

↓

Shutdown MongoDB Instance

This flow shows how MongoDB starts, initializes WiredTiger, manages memory cache, reads/writes data, and serves queries.

Execution Sample

MongoDB

1. Start MongoDB with WiredTiger
2. WiredTiger allocates cache memory
3. Data is read/written using cache
4. Cache flushes data to disk
5. Queries use cache and disk data

This sequence shows how WiredTiger manages memory and storage during MongoDB operations.

Execution Table

Step	Action	Memory Cache State	Disk State	Result
1	Start MongoDB with WiredTiger	Empty	Empty	WiredTiger engine initialized
2	Allocate cache memory (default 50% RAM)	Cache allocated (e.g. 2GB)	Empty	Memory ready for data
3	Write data to collection	Data stored in cache	Disk unchanged	Fast write acknowledged
4	Cache flush triggered	Cache partially cleared	Data written to disk	Data safely stored on disk
5	Read query requested	Data checked in cache	If missing, read from disk	Query served quickly
6	Shutdown MongoDB	Cache flushed fully	All data on disk	Clean shutdown
7	End	Cache empty	Data persisted	MongoDB stopped

💡 MongoDB stops after flushing all cache data to disk to ensure data durability.

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	Final
Memory Cache	Empty	Allocated (2GB)	Contains new data	Partially cleared	Checked for reads	Flushed fully	Empty
Disk Storage	Empty	Empty	Empty	Data written	Data read if needed	All data persisted	Data persisted

Key Moments - 3 Insights

Why does WiredTiger use a memory cache before writing to disk?

WiredTiger uses memory cache to speed up data operations by temporarily holding data in fast memory before writing it slower to disk, as shown in steps 3 and 4 of the execution_table.

What happens if data is not found in the cache during a read?

If data is missing in cache, WiredTiger reads it from disk to serve the query, as shown in step 5 of the execution_table.

Why must the cache be fully flushed before MongoDB shuts down?

Flushing the cache ensures all data is safely stored on disk to prevent data loss, as shown in step 6 of the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table, what is the state of the memory cache after step 3?

A Cache contains new data

B Cache is empty

C Cache is fully flushed

D Cache is partially cleared

💡 Hint

Check the 'Memory Cache State' column at step 3 in the execution_table.

At which step does WiredTiger write data from cache to disk?

A Step 2

B Step 4

C Step 5

D Step 6

💡 Hint

Look for 'Cache flush triggered' and 'Data written to disk' in the execution_table.

If the cache was never flushed, what would happen at shutdown (step 6)?

A MongoDB would not start

B Data would be safely stored on disk

C Data would be lost because it is only in cache

D Cache would automatically clear without writing

💡 Hint

Refer to the importance of flushing cache before shutdown in key_moments and execution_table step 6.

Concept Snapshot

WiredTiger is MongoDB's default storage engine.
It uses a memory cache to speed up reads/writes.
Data is first stored in cache, then flushed to disk.
Cache size defaults to about 50% of RAM.
Flushing cache ensures data durability on shutdown.

Full Transcript

This visual execution shows how MongoDB uses the WiredTiger storage engine to manage memory and disk storage. When MongoDB starts, WiredTiger initializes and allocates a memory cache, typically half of the system RAM. When data is written, it first goes into this fast cache, making writes quick. Periodically, WiredTiger flushes the cache, writing data safely to disk. Reads check the cache first, then disk if needed, speeding up queries. Before MongoDB shuts down, the cache is fully flushed to ensure no data is lost. This process balances speed and durability by using memory and disk efficiently.

Practice

1. What is the default storage engine used by MongoDB that balances speed and storage?

easy

A. WiredTiger

B. MMAPv1

C. InMemory

D. RocksDB

Solution

1. Step 1: Recall MongoDB default engine

   MongoDB uses WiredTiger as its default storage engine since version 3.2 for better performance and compression.

2. Step 2: Compare options

   MMAPv1 is older, InMemory is for volatile data, RocksDB is not default in MongoDB.

3. Final Answer:

   WiredTiger -> Option A

4. Quick Check:

   Default engine = WiredTiger [OK]

Hint: Default MongoDB engine is WiredTiger [OK]

Common Mistakes:

• Confusing MMAPv1 as default
• Thinking InMemory is default
• Assuming RocksDB is default

2. Which configuration file setting correctly specifies WiredTiger as the storage engine in mongod.conf?

easy

A. storage:
engine: wiredtiger

B. storage:
engine: WT

C. storage:
engine: wiredTiger

D. storage:
engine: WiredTiger

Solution

1. Step 1: Check correct case for engine name

   The engine name is case-sensitive and must be 'wiredTiger' with lowercase 'w' and capital 'T'.

2. Step 2: Validate YAML syntax

   storage: engine: WT uses unrecognized abbreviation; C uses all lowercase; D uses incorrect capitalization.

3. Final Answer:

   storage: engine: wiredTiger -> Option C

4. Quick Check:

   Correct engine name is wiredTiger [OK]

Hint: Use exact 'wiredTiger' (lowercase w, capital T) in config [OK]

Common Mistakes:

• Using all lowercase 'wiredtiger'
• Using 'WiredTiger' with capital W
• Using abbreviations like 'WT'

3. Given this mongod.conf snippet:

storage:
  engine: WiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 2

What does the cacheSizeGB setting control?

medium

A. The disk space allocated for WiredTiger files

B. The amount of RAM WiredTiger uses for caching data

C. The number of CPU cores WiredTiger uses

D. The network bandwidth limit for WiredTiger

Solution

1. Step 1: Understand cacheSizeGB meaning

   cacheSizeGB sets how much RAM WiredTiger uses to cache data in memory for faster access.

2. Step 2: Eliminate other options

   It does not control disk space, CPU cores, or network bandwidth.

3. Final Answer:

   The amount of RAM WiredTiger uses for caching data -> Option B

4. Quick Check:

   cacheSizeGB = RAM cache size [OK]

Hint: cacheSizeGB sets RAM cache size for WiredTiger [OK]

Common Mistakes:

• Thinking it controls disk space
• Confusing with CPU or network settings
• Ignoring memory tuning importance

4. You set cacheSizeGB: 0.5 in mongod.conf but MongoDB starts with poor performance. What is the likely cause?

medium

A. cacheSizeGB must be at least 1 GB

B. cacheSizeGB cannot be a decimal value

C. cacheSizeGB must be set under storage.engine directly

D. cacheSizeGB is ignored and does not affect startup

Solution

1. Step 1: Check WiredTiger cache size limits

   WiredTiger performs poorly if cacheSizeGB is less than 1 GB.

2. Step 2: Validate config placement and value type

   Decimal values are allowed; placement under wiredTiger.engineConfig is correct; it is not ignored.

3. Final Answer:

   cacheSizeGB must be at least 1 GB -> Option A

4. Quick Check:

   Minimum cacheSizeGB = 1 GB [OK]

Hint: cacheSizeGB minimum is 1 GB for good performance [OK]

Common Mistakes:

• Using decimal values thinking they are invalid
• Placing cacheSizeGB under wrong config section
• Assuming cacheSizeGB is optional for startup

5. You want to optimize MongoDB performance on a server with 16 GB RAM. Which cacheSizeGB setting is best for WiredTiger, considering MongoDB and OS need memory?

hard

A. Set cacheSizeGB to 14 to maximize cache

B. Set cacheSizeGB to 16 to use all RAM for cache

C. Set cacheSizeGB to 2 because more cache slows down MongoDB

D. Set cacheSizeGB to 8 to leave memory for OS and other processes

Solution

1. Step 1: Understand memory needs

   MongoDB needs memory for cache and other operations; OS and other apps also need RAM.

2. Step 2: Choose balanced cache size

   Setting cacheSizeGB to about half or slightly more (8 GB) leaves enough RAM for OS and avoids swapping.

3. Step 3: Evaluate other options

   14 or 16 GB risks OS starvation; 2 GB is too low for good performance.

4. Final Answer:

   Set cacheSizeGB to 8 to leave memory for OS and other processes -> Option D

5. Quick Check:

   Balance cache and OS memory = 8 GB [OK]

Hint: Use about half RAM for cache to keep system stable [OK]

Common Mistakes:

• Assigning all RAM to cache causing OS issues
• Setting too low cacheSizeGB hurting performance
• Ignoring OS memory needs