Saving and loading data (scipy.io) - Time & Space Complexity

Time Complexity: Saving and loading data (scipy.io)

O(n)

Understanding Time Complexity

When saving or loading data with scipy.io, we want to know how the time needed changes as the data size grows.

We ask: How does the time to save or load data grow when the data gets bigger?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import numpy as np
from scipy import io

data = np.random.rand(1000, 1000)  # Create a large array
io.savemat('datafile.mat', {'array': data})  # Save data to a .mat file
loaded = io.loadmat('datafile.mat')  # Load data back from the file

This code creates a large array, saves it to a file, and then loads it back into memory.

Identify Repeating Operations

• Primary operation: Reading or writing each element of the array to or from disk.
• How many times: Once for each element in the array (all 1,000,000 elements).

How Execution Grows With Input

As the data size grows, the time to save or load grows roughly in proportion to the number of elements.

Input Size (n x n)	Approx. Operations
10 x 10	100
100 x 100	10,000
1000 x 1000	1,000,000

Pattern observation: Doubling the size in each dimension multiplies the total operations by the square, so time grows linearly with total elements.

Final Time Complexity

Time Complexity: O(n)

This means the time to save or load data grows directly with the number of elements in the data.

Common Mistake

[X] Wrong: "Saving or loading data takes the same time no matter how big the data is."

[OK] Correct: The time depends on how many elements are saved or loaded, so bigger data takes more time.

Interview Connect

Understanding how saving and loading time grows helps you handle large datasets efficiently and shows you know how data size affects performance.

Self-Check

"What if we compressed the data before saving? How would the time complexity change?"

Practice

1. What is the primary purpose of scipy.io.savemat in data science?

easy

A. To load data from a CSV file

B. To save Python variables into a MATLAB .mat file

C. To visualize data in plots

D. To convert Python code to MATLAB code

Solution

1. Step 1: Understand the function purpose

   scipy.io.savemat is designed to save Python data into MATLAB's .mat file format.

2. Step 2: Compare options with function use

   Only To save Python variables into a MATLAB .mat file correctly describes saving Python variables into a .mat file. Other options describe unrelated tasks.

3. Final Answer:

   To save Python variables into a MATLAB .mat file -> Option B

4. Quick Check:

   savemat saves data = A [OK]

Hint: Remember: savemat saves, loadmat loads [OK]

Common Mistakes:

• Confusing savemat with loadmat
• Thinking savemat loads data
• Assuming it works with CSV files

2. Which of the following is the correct way to load a .mat file named data.mat using scipy?

easy

A. data = io.loadmat('data.mat')

B. data = io.savemat('data.mat')

C. data = io.load('data.mat')

D. data = io.readmat('data.mat')

Solution

1. Step 1: Identify the correct function for loading .mat files

   The function to load MATLAB files in scipy is loadmat.

2. Step 2: Check syntax correctness

   data = io.loadmat('data.mat') uses io.loadmat('data.mat'), which is the correct syntax. Other options use incorrect function names.

3. Final Answer:

   data = io.loadmat('data.mat') -> Option A

4. Quick Check:

   loadmat loads .mat files = A [OK]

Hint: Load with loadmat, save with savemat [OK]

Common Mistakes:

• Using savemat to load data
• Using non-existent functions like readmat
• Missing parentheses or quotes

3. What will be the output of this code snippet?

import numpy as np
from scipy import io
arr = np.array([1, 2, 3])
io.savemat('test.mat', {'array': arr})
data = io.loadmat('test.mat')
print(data['array'])

medium

A. [[1 2 3]]

B. Error: KeyError

C. [[1]
[2]
[3]]

D. [1 2 3]

Solution

1. Step 1: Understand how savemat stores arrays

   When saving a 1D numpy array, savemat stores it as a 2D array with shape (1, n) by default.

2. Step 2: Check the loaded data shape

   Loading back with loadmat returns a 2D array with shape (1, 3), so printing shows [[1 2 3]].

3. Final Answer:

   [[1 2 3]] -> Option A

4. Quick Check:

   1D array saved as 2D row = [[1 2 3]] [OK]

Hint: Loaded arrays from .mat are often 2D, not 1D [OK]

Common Mistakes:

• Expecting 1D array output
• Confusing row vs column shape
• Assuming KeyError due to wrong key

4. Identify the error in this code snippet:

from scipy import io
my_data = {'x': [1, 2, 3]}
io.savemat('file.mat', my_data)
loaded = io.loadmat('file.mat')
print(loaded['my_data'])

medium

A. FileNotFoundError when loading file.mat

B. SyntaxError in savemat call

C. KeyError because 'my_data' is not a key in loaded dict

D. TypeError because list cannot be saved

Solution

1. Step 1: Check keys saved in .mat file

   The dictionary key 'x' is saved, not the variable name 'my_data'. So loaded dict has key 'x', not 'my_data'.

2. Step 2: Understand the cause of KeyError

   Trying to access loaded['my_data'] causes KeyError because that key does not exist.

3. Final Answer:

   KeyError because 'my_data' is not a key in loaded dict -> Option C

4. Quick Check:

   Access saved keys, not variable names [OK]

Hint: Access keys used in savemat dict, not variable names [OK]

Common Mistakes:

• Assuming variable name is key in loaded dict
• Confusing syntax errors with runtime errors
• Expecting automatic key renaming

5. You want to save multiple numpy arrays a and b into a single .mat file and later load them back. Which code correctly saves and loads these arrays so you can access them by their variable names?

hard

A. io.savemat('multi.mat', [a, b])
data = io.loadmat('multi.mat')
print(data['a'], data['b'])

B. io.savemat('multi.mat', a, b)
data = io.loadmat('multi.mat')
print(data['a'], data['b'])

C. io.savemat('multi.mat', {'a': a, 'b': b})
data = io.loadmat('multi.mat')
print(data['multi']['a'], data['multi']['b'])

D. io.savemat('multi.mat', {'a': a, 'b': b})
data = io.loadmat('multi.mat')
print(data['a'], data['b'])

Solution

1. Step 1: Save multiple arrays with a dictionary

   Use a dictionary with keys as variable names and values as arrays in savemat. io.savemat('multi.mat', {'a': a, 'b': b}) data = io.loadmat('multi.mat') print(data['a'], data['b']) does this correctly.

2. Step 2: Load and access arrays by keys

   After loading, access arrays by their keys 'a' and 'b' in the loaded dictionary. io.savemat('multi.mat', {'a': a, 'b': b}) data = io.loadmat('multi.mat') print(data['a'], data['b']) prints them correctly.

3. Final Answer:

   io.savemat('multi.mat', {'a': a, 'b': b}) data = io.loadmat('multi.mat') print(data['a'], data['b']) -> Option D

4. Quick Check:

   Save/load dict with keys = variable access [OK]

Hint: Save dict with names, load dict and access keys [OK]

Common Mistakes:

• Passing list instead of dict to savemat
• Trying to access nested keys incorrectly
• Passing multiple args to savemat instead of one dict