Sparse matrix file I/O in SciPy - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of saving and loading a sparse matrix

What will be the output shape of the sparse matrix after saving and loading it using scipy's save_npz and load_npz?

SciPy

import numpy as np
from scipy.sparse import csr_matrix, save_npz, load_npz

matrix = csr_matrix(np.array([[0, 0, 1], [1, 0, 0], [0, 2, 0]]))
save_npz('test_matrix.npz', matrix)
loaded_matrix = load_npz('test_matrix.npz')
print(loaded_matrix.shape)

A (3, 2)

B (2, 3)

C (3, 3)

D (2, 2)

💡 Hint

The shape of the matrix does not change when saving and loading with scipy sparse functions.

The save_npz and load_npz functions preserve the sparse matrix structure and shape exactly. The original matrix is 3 rows by 3 columns, so the loaded matrix shape is (3, 3).

❓ data_output

intermediate

Data content after loading sparse matrix

After saving and loading the sparse matrix below, what will be the value at position (1, 0) in the loaded matrix?

SciPy

import numpy as np
from scipy.sparse import csr_matrix, save_npz, load_npz

matrix = csr_matrix(np.array([[0, 0, 1], [1, 0, 0], [0, 2, 0]]))
save_npz('test_matrix.npz', matrix)
loaded_matrix = load_npz('test_matrix.npz')
print(loaded_matrix[1, 0])

A None

B 1

C 0

D 2

💡 Hint

Check the original matrix value at row 1, column 0.

The original matrix has value 1 at position (1, 0). Saving and loading does not change values, so the loaded matrix at (1, 0) is 1.

🔧 Debug

advanced

Identify the error when loading a sparse matrix

What error will occur when trying to load a sparse matrix from a file that does not exist using load_npz?

SciPy

from scipy.sparse import load_npz
loaded_matrix = load_npz('nonexistent_file.npz')

A TypeError

B ValueError

C KeyError

D FileNotFoundError

💡 Hint

Think about what happens when you try to open a file that does not exist.

Trying to load a file that does not exist raises a FileNotFoundError because the file system cannot find the specified file.

🚀 Application

advanced

Choosing the best file format for sparse matrix storage

You have a large sparse matrix that you want to save and share with others. Which file format is best to use for saving and loading sparse matrices efficiently with scipy?

A NPZ format using save_npz and load_npz

B CSV format using numpy.savetxt and numpy.loadtxt

C TXT format using Python's open and write

D JSON format using json.dump and json.load

💡 Hint

Consider which format preserves sparsity and is optimized for scipy sparse matrices.

NPZ format with save_npz and load_npz is designed for sparse matrices. It preserves sparsity and is efficient in storage and speed. CSV, TXT, and JSON do not preserve sparsity and are inefficient for large sparse data.

🧠 Conceptual

expert

Understanding sparse matrix file I/O internals

Which of the following best describes what save_npz stores when saving a scipy CSR sparse matrix?

A It stores the data array, indices array, indptr array, and shape tuple in a compressed NPZ file

B It stores the full dense matrix as a numpy array in the NPZ file

C It stores only the nonzero values in a plain text file

D It stores the matrix as a JSON object with keys for rows and columns

💡 Hint

Think about how CSR format represents sparse matrices internally.

The CSR format stores three arrays: data (nonzero values), indices (column indices), and indptr (index pointers for row starts), plus the shape. save_npz saves these arrays and shape in a compressed NPZ file for efficient storage.

Practice

1. What is the main purpose of using save_npz and load_npz functions in scipy.sparse?

easy

A. To perform matrix multiplication on sparse matrices

B. To convert sparse matrices into dense matrices

C. To visualize sparse matrices as heatmaps

D. To save and load sparse matrices efficiently without losing their structure

Solution

1. Step 1: Understand the purpose of save_npz and load_npz

   These functions are designed to save sparse matrices to disk and load them back while keeping their sparse format intact.

2. Step 2: Compare options with the purpose

   Only To save and load sparse matrices efficiently without losing their structure correctly describes saving and loading sparse matrices efficiently without losing their sparse structure.

3. Final Answer:

   To save and load sparse matrices efficiently without losing their structure -> Option D

4. Quick Check:

   Sparse matrix file I/O = save/load sparse matrices [OK]

Hint: Remember save_npz/load_npz keep sparse format intact [OK]

Common Mistakes:

• Thinking these functions convert to dense matrices
• Confusing file I/O with matrix operations
• Assuming visualization is part of file I/O

2. Which of the following is the correct way to save a sparse matrix sp_matrix to a file named data.npz using SciPy?

easy

A. scipy.sparse.save('data.npz', sp_matrix)

B. scipy.sparse.save_npz('data.npz', sp_matrix)

C. scipy.sparse.load_npz('data.npz', sp_matrix)

D. scipy.save_npz(sp_matrix, 'data.npz')

Solution

1. Step 1: Identify the correct function and argument order

   The function to save sparse matrices is save_npz from scipy.sparse, and it takes the filename first, then the matrix.

2. Step 2: Check each option

   scipy.sparse.save_npz('data.npz', sp_matrix) matches the correct syntax: save_npz('filename', matrix). Others either use wrong function names or argument order.

3. Final Answer:

   scipy.sparse.save_npz('data.npz', sp_matrix) -> Option B

4. Quick Check:

   save_npz(filename, matrix) = scipy.sparse.save_npz('data.npz', sp_matrix) [OK]

Hint: save_npz(filename, matrix) saves sparse matrix [OK]

Common Mistakes:

• Using load_npz instead of save_npz to save
• Swapping filename and matrix arguments
• Using non-existent save function

3. Consider the following code snippet:

from scipy.sparse import csr_matrix, save_npz, load_npz
import numpy as np

arr = np.array([[0, 0, 1], [1, 0, 0], [0, 2, 0]])
sp = csr_matrix(arr)
save_npz('matrix.npz', sp)
loaded_sp = load_npz('matrix.npz')
print(loaded_sp.toarray())

What will be the output printed?

medium

A. Error: cannot convert sparse matrix to array

B. [[0 0 0]
[0 0 0]
[0 0 0]]

C. [[0 0 1]
[1 0 0]
[0 2 0]]

D. [[1 0 0]
[0 1 0]
[0 0 1]]

Solution

1. Step 1: Create sparse matrix and save it

   The code creates a sparse matrix from the numpy array, saves it to 'matrix.npz', then loads it back.

2. Step 2: Convert loaded sparse matrix to dense array and print

   Using toarray() converts the sparse matrix back to the original dense numpy array, so the printed output matches the original array.

3. Final Answer:

   [[0 0 1] [1 0 0] [0 2 0]] -> Option C

4. Quick Check:

   load_npz + toarray() = original array [OK]

Hint: load_npz returns sparse; use toarray() to see full matrix [OK]

Common Mistakes:

• Expecting zeros after loading
• Forgetting to convert sparse to dense before printing
• Confusing save_npz and load_npz usage

4. You wrote this code to load a sparse matrix:

from scipy.sparse import load_npz
matrix = load_npz('data.npz')
print(matrix)

But you get an error: ModuleNotFoundError: No module named 'scipy.sparse'. What is the most likely cause?

medium

A. You forgot to install the SciPy library in your environment

B. The file 'data.npz' does not exist

C. You used load_npz instead of save_npz

D. You need to convert the matrix to dense before printing

Solution

1. Step 1: Analyze the error message

   The error says the module 'scipy.sparse' is not found, which means SciPy is not installed or not accessible.

2. Step 2: Check other options

   File missing causes a different error, wrong function usage or printing sparse matrix won't cause module import error.

3. Final Answer:

   You forgot to install the SciPy library in your environment -> Option A

4. Quick Check:

   ModuleNotFoundError = missing SciPy install [OK]

Hint: ModuleNotFoundError means missing package install [OK]

Common Mistakes:

• Assuming file missing causes import error
• Confusing function usage errors with import errors
• Thinking sparse matrix print needs conversion to avoid import error

5. You have a large sparse matrix stored in large_matrix.npz. You want to load it, add 5 to all non-zero elements, and save it back without converting to a dense matrix (to save memory). Which code snippet correctly does this?

hard

A. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat.data += 5
save_npz('large_matrix.npz', mat)

B. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat = mat.toarray() + 5
save_npz('large_matrix.npz', mat)

C. import numpy as np
mat = np.load('large_matrix.npz')
mat += 5
np.save('large_matrix.npz', mat)

D. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat.toarray() += 5
save_npz('large_matrix.npz', mat)

Solution

1. Step 1: Load sparse matrix and modify non-zero elements

   Using mat.data accesses the non-zero values directly. Adding 5 to mat.data updates only those values without converting to dense.

2. Step 2: Save the updated sparse matrix back

   Using save_npz saves the modified sparse matrix efficiently.

3. Step 3: Check other options for correctness

   from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat = mat.toarray() + 5 save_npz('large_matrix.npz', mat) converts to dense explicitly, adds 5 to all elements including zeros, wastes memory, and save_npz fails on dense array. import numpy as np mat = np.load('large_matrix.npz') mat += 5 np.save('large_matrix.npz', mat) uses numpy load/save which does not handle sparse matrices. from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat.toarray() += 5 save_npz('large_matrix.npz', mat) tries to add 5 to a dense array but does not assign back, and wastes memory.

4. Final Answer:

   from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat.data += 5 save_npz('large_matrix.npz', mat) -> Option A

5. Quick Check:

   Modify mat.data for sparse update [OK]

Hint: Change mat.data to update non-zero sparse values [OK]

Common Mistakes:

• Adding scalar directly to sparse matrix (converts to dense)
• Using numpy load/save for sparse matrices
• Modifying dense array without saving changes