Memory-mapped files with np.memmap in NumPy - Step-by-Step Execution

Concept Flow - Memory-mapped files with np.memmap

Create memmap file

↓

Access data via memmap

↓

Read/Write data

↓

Changes saved to disk

↓

Close memmap file

Memory-mapped files let you work with large arrays stored on disk as if they were in memory, reading and writing parts without loading all at once.

Execution Sample

NumPy

import numpy as np
filename = 'data.dat'
# Create memmap
mmap = np.memmap(filename, dtype='float32', mode='w+', shape=(3,3))
mmap[:] = np.arange(9).reshape(3,3)
mmap.flush()

This code creates a 3x3 memory-mapped file, writes numbers 0 to 8 into it, and saves changes to disk.

Execution Table

Step	Action	Variable/Expression	Result/State
1	Create memmap file	np.memmap(filename, dtype='float32', mode='w+', shape=(3,3))	3x3 array on disk, uninitialized
2	Assign values	mmap[:] = np.arange(9).reshape(3,3)	mmap contains [[0,1,2],[3,4,5],[6,7,8]]
3	Flush changes	mmap.flush()	Data written to 'data.dat' on disk
4	Read value	mmap[1,2]	5.0
5	Modify value	mmap[0,0] = 100	mmap updated at position (0,0)
6	Flush changes	mmap.flush()	Changes saved to disk
7	Close memmap	del mmap	Memory map closed, file remains on disk

💡 Finished writing and reading data; memmap closed to release resources.

Variable Tracker

Variable	Start	After Step 2	After Step 5	Final
mmap	empty 3x3 array on disk	[[0,1,2],[3,4,5],[6,7,8]]	[[100,1,2],[3,4,5],[6,7,8]]	same as after step 5, closed after step 7

Key Moments - 3 Insights

Why do we need to call mmap.flush() after modifying data?

Calling mmap.flush() writes changes from memory back to the disk file. Without it, changes may stay only in memory and not be saved, as shown in steps 3 and 6.

Is the entire file loaded into memory when creating a memmap?

No, memmap loads only parts of the file as needed, allowing working with large files without using much RAM. Step 1 creates the file but does not load all data.

What happens if we delete the mmap variable?

Deleting mmap closes the memory map and releases resources, but the file remains on disk with saved data, as in step 7.

Visual Quiz - 3 Questions

Look at the execution table, what is the value of mmap[1,2] at step 4?

A 4

B 5

C 2

D 100

💡 Hint

Check step 4 in the execution_table where mmap[1,2] is read.

At which step does the value at position (0,0) change to 100?

A Step 2

B Step 3

C Step 5

D Step 6

💡 Hint

Look at the 'Modify value' action in the execution_table.

If we skip calling mmap.flush() after modifying data, what happens?

A Changes remain only in memory and may be lost

B File is deleted

C Changes are saved immediately to disk

D Program crashes

💡 Hint

Refer to key_moments about why flush() is needed.

Concept Snapshot

np.memmap(filename, dtype, mode, shape) creates a memory-mapped array.
You can read/write like a normal array.
Changes are saved to disk with flush().
Good for large data that doesn't fit in RAM.
Close or delete memmap to release resources.

Full Transcript

Memory-mapped files with np.memmap allow working with large arrays stored on disk as if they were in memory. You create a memmap object pointing to a file, then read or write data like a normal array. Changes are not saved automatically; you must call flush() to write them to disk. This method helps handle big data without loading it all into RAM. When done, delete or close the memmap to free resources. The execution steps show creating a 3x3 memmap, writing numbers, reading a value, modifying it, flushing changes, and closing the map.