Sparse matrix file I/O in SciPy - Mini Project: Build & Apply

Sparse matrix file I/O

📖 Scenario: You work with large datasets that have many zero values. To save space, you use sparse matrices. You want to save your sparse matrix to a file and then load it back later.

🎯 Goal: Create a sparse matrix, save it to a file, load it back from the file, and print the loaded matrix.

📋 What You'll Learn

Use scipy.sparse to create a sparse matrix

Save the sparse matrix to a file using scipy.sparse.save_npz

Load the sparse matrix from the file using scipy.sparse.load_npz

Print the loaded sparse matrix

💡 Why This Matters

🌍 Real World

Sparse matrices are used in machine learning, scientific computing, and data storage when data has many zeros. Saving and loading them efficiently saves disk space and speeds up processing.

💼 Career

Data scientists and engineers often work with large sparse datasets. Knowing how to save and load sparse matrices is important for managing data pipelines and model training.

1

Create a sparse matrix

Import scipy.sparse and create a 3x3 sparse matrix called matrix with these values: 1 at (0,0), 2 at (1,1), and 3 at (2,2) using scipy.sparse.csr_matrix.

SciPy

# Import scipy.sparse and create the sparse matrix called matrix
# Your code here

Hint

Use scipy.sparse.csr_matrix with a list of lists to create the sparse matrix.

2

Set the filename for saving

Create a variable called filename and set it to the string 'matrix.npz' to use as the file name for saving the sparse matrix.

SciPy

import scipy.sparse
matrix = scipy.sparse.csr_matrix([[1, 0, 0], [0, 2, 0], [0, 0, 3]])
# Create the filename variable
# Your code here

Hint

Just assign the string 'matrix.npz' to the variable filename.

3

Save and load the sparse matrix

Use scipy.sparse.save_npz to save matrix to filename. Then use scipy.sparse.load_npz to load the matrix back into a variable called loaded_matrix.

SciPy

import scipy.sparse
matrix = scipy.sparse.csr_matrix([[1, 0, 0], [0, 2, 0], [0, 0, 3]])
filename = 'matrix.npz'
# Save matrix to filename and load it back into loaded_matrix
# Your code here

Hint

Use scipy.sparse.save_npz(filename, matrix) to save and loaded_matrix = scipy.sparse.load_npz(filename) to load.

4

Print the loaded sparse matrix

Print the variable loaded_matrix to display the loaded sparse matrix.

SciPy

import scipy.sparse
matrix = scipy.sparse.csr_matrix([[1, 0, 0], [0, 2, 0], [0, 0, 3]])
filename = 'matrix.npz'
scipy.sparse.save_npz(filename, matrix)
loaded_matrix = scipy.sparse.load_npz(filename)
# Print the loaded_matrix
# Your code here

Hint

Use print(loaded_matrix) to show the sparse matrix.

Practice

1. What is the main purpose of using save_npz and load_npz functions in scipy.sparse?

easy

A. To perform matrix multiplication on sparse matrices

B. To convert sparse matrices into dense matrices

C. To visualize sparse matrices as heatmaps

D. To save and load sparse matrices efficiently without losing their structure

Solution

1. Step 1: Understand the purpose of save_npz and load_npz

   These functions are designed to save sparse matrices to disk and load them back while keeping their sparse format intact.

2. Step 2: Compare options with the purpose

   Only To save and load sparse matrices efficiently without losing their structure correctly describes saving and loading sparse matrices efficiently without losing their sparse structure.

3. Final Answer:

   To save and load sparse matrices efficiently without losing their structure -> Option D

4. Quick Check:

   Sparse matrix file I/O = save/load sparse matrices [OK]

Hint: Remember save_npz/load_npz keep sparse format intact [OK]

Common Mistakes:

• Thinking these functions convert to dense matrices
• Confusing file I/O with matrix operations
• Assuming visualization is part of file I/O

2. Which of the following is the correct way to save a sparse matrix sp_matrix to a file named data.npz using SciPy?

easy

A. scipy.sparse.save('data.npz', sp_matrix)

B. scipy.sparse.save_npz('data.npz', sp_matrix)

C. scipy.sparse.load_npz('data.npz', sp_matrix)

D. scipy.save_npz(sp_matrix, 'data.npz')

Solution

1. Step 1: Identify the correct function and argument order

   The function to save sparse matrices is save_npz from scipy.sparse, and it takes the filename first, then the matrix.

2. Step 2: Check each option

   scipy.sparse.save_npz('data.npz', sp_matrix) matches the correct syntax: save_npz('filename', matrix). Others either use wrong function names or argument order.

3. Final Answer:

   scipy.sparse.save_npz('data.npz', sp_matrix) -> Option B

4. Quick Check:

   save_npz(filename, matrix) = scipy.sparse.save_npz('data.npz', sp_matrix) [OK]

Hint: save_npz(filename, matrix) saves sparse matrix [OK]

Common Mistakes:

• Using load_npz instead of save_npz to save
• Swapping filename and matrix arguments
• Using non-existent save function

3. Consider the following code snippet:

from scipy.sparse import csr_matrix, save_npz, load_npz
import numpy as np

arr = np.array([[0, 0, 1], [1, 0, 0], [0, 2, 0]])
sp = csr_matrix(arr)
save_npz('matrix.npz', sp)
loaded_sp = load_npz('matrix.npz')
print(loaded_sp.toarray())

What will be the output printed?

medium

A. Error: cannot convert sparse matrix to array

B. [[0 0 0]
[0 0 0]
[0 0 0]]

C. [[0 0 1]
[1 0 0]
[0 2 0]]

D. [[1 0 0]
[0 1 0]
[0 0 1]]

Solution

1. Step 1: Create sparse matrix and save it

   The code creates a sparse matrix from the numpy array, saves it to 'matrix.npz', then loads it back.

2. Step 2: Convert loaded sparse matrix to dense array and print

   Using toarray() converts the sparse matrix back to the original dense numpy array, so the printed output matches the original array.

3. Final Answer:

   [[0 0 1] [1 0 0] [0 2 0]] -> Option C

4. Quick Check:

   load_npz + toarray() = original array [OK]

Hint: load_npz returns sparse; use toarray() to see full matrix [OK]

Common Mistakes:

• Expecting zeros after loading
• Forgetting to convert sparse to dense before printing
• Confusing save_npz and load_npz usage

4. You wrote this code to load a sparse matrix:

from scipy.sparse import load_npz
matrix = load_npz('data.npz')
print(matrix)

But you get an error: ModuleNotFoundError: No module named 'scipy.sparse'. What is the most likely cause?

medium

A. You forgot to install the SciPy library in your environment

B. The file 'data.npz' does not exist

C. You used load_npz instead of save_npz

D. You need to convert the matrix to dense before printing

Solution

1. Step 1: Analyze the error message

   The error says the module 'scipy.sparse' is not found, which means SciPy is not installed or not accessible.

2. Step 2: Check other options

   File missing causes a different error, wrong function usage or printing sparse matrix won't cause module import error.

3. Final Answer:

   You forgot to install the SciPy library in your environment -> Option A

4. Quick Check:

   ModuleNotFoundError = missing SciPy install [OK]

Hint: ModuleNotFoundError means missing package install [OK]

Common Mistakes:

• Assuming file missing causes import error
• Confusing function usage errors with import errors
• Thinking sparse matrix print needs conversion to avoid import error

5. You have a large sparse matrix stored in large_matrix.npz. You want to load it, add 5 to all non-zero elements, and save it back without converting to a dense matrix (to save memory). Which code snippet correctly does this?

hard

A. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat.data += 5
save_npz('large_matrix.npz', mat)

B. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat = mat.toarray() + 5
save_npz('large_matrix.npz', mat)

C. import numpy as np
mat = np.load('large_matrix.npz')
mat += 5
np.save('large_matrix.npz', mat)

D. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat.toarray() += 5
save_npz('large_matrix.npz', mat)

Solution

1. Step 1: Load sparse matrix and modify non-zero elements

   Using mat.data accesses the non-zero values directly. Adding 5 to mat.data updates only those values without converting to dense.

2. Step 2: Save the updated sparse matrix back

   Using save_npz saves the modified sparse matrix efficiently.

3. Step 3: Check other options for correctness

   from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat = mat.toarray() + 5 save_npz('large_matrix.npz', mat) converts to dense explicitly, adds 5 to all elements including zeros, wastes memory, and save_npz fails on dense array. import numpy as np mat = np.load('large_matrix.npz') mat += 5 np.save('large_matrix.npz', mat) uses numpy load/save which does not handle sparse matrices. from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat.toarray() += 5 save_npz('large_matrix.npz', mat) tries to add 5 to a dense array but does not assign back, and wastes memory.

4. Final Answer:

   from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat.data += 5 save_npz('large_matrix.npz', mat) -> Option A

5. Quick Check:

   Modify mat.data for sparse update [OK]

Hint: Change mat.data to update non-zero sparse values [OK]

Common Mistakes:

• Adding scalar directly to sparse matrix (converts to dense)
• Using numpy load/save for sparse matrices
• Modifying dense array without saving changes