Sparse matrix file I/O in SciPy

Introduction

Sparse matrices save memory by storing mostly zeros efficiently. Saving and loading them helps keep data safe and reuse it later.

You want to save a large sparse matrix after a long calculation to avoid repeating it.

You need to share a sparse matrix with a teammate or another program.

You want to load a sparse matrix from disk to analyze or visualize it.

You are working with machine learning data that is mostly zeros and want to store it efficiently.

Syntax

SciPy

from scipy import sparse

# Save sparse matrix
sparse.save_npz(filename, sparse_matrix)

# Load sparse matrix
loaded_matrix = sparse.load_npz(filename)

Use save_npz to save and load_npz to load sparse matrices in compressed format.

This works with common sparse formats like CSR and CSC.

Examples

Create a CSR sparse matrix and save it to 'matrix.npz'.

SciPy

from scipy import sparse
import numpy as np

# Create a sparse matrix
matrix = sparse.csr_matrix(np.array([[0, 0, 1], [1, 0, 0], [0, 0, 0]]))

# Save it
sparse.save_npz('matrix.npz', matrix)

Load the saved sparse matrix and convert it to a normal array to see the data.

SciPy

from scipy import sparse

# Load the sparse matrix
loaded = sparse.load_npz('matrix.npz')
print(loaded.toarray())

Sample Program

This program creates a sparse matrix, saves it to a file, loads it back, and prints the full matrix to verify it saved correctly.

SciPy

from scipy import sparse
import numpy as np

# Create a sparse matrix with mostly zeros
rows = np.array([0, 1, 2])
cols = np.array([2, 0, 1])
data = np.array([1, 2, 3])

matrix = sparse.csr_matrix((data, (rows, cols)), shape=(3, 3))

# Save the sparse matrix to a file
sparse.save_npz('example_matrix.npz', matrix)

# Load the sparse matrix back
loaded_matrix = sparse.load_npz('example_matrix.npz')

# Print the dense form to check
print(loaded_matrix.toarray())

Important Notes

Always use toarray() or todense() to see the full matrix when printing.

File extension .npz is recommended for saving sparse matrices.

Saving in sparse format is much faster and smaller than saving dense matrices when data is mostly zeros.

Summary

Sparse matrix file I/O lets you save and load memory-efficient matrices easily.

Use save_npz and load_npz from scipy.sparse.

This helps reuse data and share it without losing the sparse structure.

Practice

1. What is the main purpose of using save_npz and load_npz functions in scipy.sparse?

easy

A. To perform matrix multiplication on sparse matrices

B. To convert sparse matrices into dense matrices

C. To visualize sparse matrices as heatmaps

D. To save and load sparse matrices efficiently without losing their structure

Solution

1. Step 1: Understand the purpose of save_npz and load_npz

   These functions are designed to save sparse matrices to disk and load them back while keeping their sparse format intact.

2. Step 2: Compare options with the purpose

   Only To save and load sparse matrices efficiently without losing their structure correctly describes saving and loading sparse matrices efficiently without losing their sparse structure.

3. Final Answer:

   To save and load sparse matrices efficiently without losing their structure -> Option D

4. Quick Check:

   Sparse matrix file I/O = save/load sparse matrices [OK]

Hint: Remember save_npz/load_npz keep sparse format intact [OK]

Common Mistakes:

• Thinking these functions convert to dense matrices
• Confusing file I/O with matrix operations
• Assuming visualization is part of file I/O

2. Which of the following is the correct way to save a sparse matrix sp_matrix to a file named data.npz using SciPy?

easy

A. scipy.sparse.save('data.npz', sp_matrix)

B. scipy.sparse.save_npz('data.npz', sp_matrix)

C. scipy.sparse.load_npz('data.npz', sp_matrix)

D. scipy.save_npz(sp_matrix, 'data.npz')

Solution

1. Step 1: Identify the correct function and argument order

   The function to save sparse matrices is save_npz from scipy.sparse, and it takes the filename first, then the matrix.

2. Step 2: Check each option

   scipy.sparse.save_npz('data.npz', sp_matrix) matches the correct syntax: save_npz('filename', matrix). Others either use wrong function names or argument order.

3. Final Answer:

   scipy.sparse.save_npz('data.npz', sp_matrix) -> Option B

4. Quick Check:

   save_npz(filename, matrix) = scipy.sparse.save_npz('data.npz', sp_matrix) [OK]

Hint: save_npz(filename, matrix) saves sparse matrix [OK]

Common Mistakes:

• Using load_npz instead of save_npz to save
• Swapping filename and matrix arguments
• Using non-existent save function

3. Consider the following code snippet:

from scipy.sparse import csr_matrix, save_npz, load_npz
import numpy as np

arr = np.array([[0, 0, 1], [1, 0, 0], [0, 2, 0]])
sp = csr_matrix(arr)
save_npz('matrix.npz', sp)
loaded_sp = load_npz('matrix.npz')
print(loaded_sp.toarray())

What will be the output printed?

medium

A. Error: cannot convert sparse matrix to array

B. [[0 0 0]
[0 0 0]
[0 0 0]]

C. [[0 0 1]
[1 0 0]
[0 2 0]]

D. [[1 0 0]
[0 1 0]
[0 0 1]]

Solution

1. Step 1: Create sparse matrix and save it

   The code creates a sparse matrix from the numpy array, saves it to 'matrix.npz', then loads it back.

2. Step 2: Convert loaded sparse matrix to dense array and print

   Using toarray() converts the sparse matrix back to the original dense numpy array, so the printed output matches the original array.

3. Final Answer:

   [[0 0 1] [1 0 0] [0 2 0]] -> Option C

4. Quick Check:

   load_npz + toarray() = original array [OK]

Hint: load_npz returns sparse; use toarray() to see full matrix [OK]

Common Mistakes:

• Expecting zeros after loading
• Forgetting to convert sparse to dense before printing
• Confusing save_npz and load_npz usage

4. You wrote this code to load a sparse matrix:

from scipy.sparse import load_npz
matrix = load_npz('data.npz')
print(matrix)

But you get an error: ModuleNotFoundError: No module named 'scipy.sparse'. What is the most likely cause?

medium

A. You forgot to install the SciPy library in your environment

B. The file 'data.npz' does not exist

C. You used load_npz instead of save_npz

D. You need to convert the matrix to dense before printing

Solution

1. Step 1: Analyze the error message

   The error says the module 'scipy.sparse' is not found, which means SciPy is not installed or not accessible.

2. Step 2: Check other options

   File missing causes a different error, wrong function usage or printing sparse matrix won't cause module import error.

3. Final Answer:

   You forgot to install the SciPy library in your environment -> Option A

4. Quick Check:

   ModuleNotFoundError = missing SciPy install [OK]

Hint: ModuleNotFoundError means missing package install [OK]

Common Mistakes:

• Assuming file missing causes import error
• Confusing function usage errors with import errors
• Thinking sparse matrix print needs conversion to avoid import error

5. You have a large sparse matrix stored in large_matrix.npz. You want to load it, add 5 to all non-zero elements, and save it back without converting to a dense matrix (to save memory). Which code snippet correctly does this?

hard

A. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat.data += 5
save_npz('large_matrix.npz', mat)

B. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat = mat.toarray() + 5
save_npz('large_matrix.npz', mat)

C. import numpy as np
mat = np.load('large_matrix.npz')
mat += 5
np.save('large_matrix.npz', mat)

D. from scipy.sparse import load_npz, save_npz
mat = load_npz('large_matrix.npz')
mat.toarray() += 5
save_npz('large_matrix.npz', mat)

Solution

1. Step 1: Load sparse matrix and modify non-zero elements

   Using mat.data accesses the non-zero values directly. Adding 5 to mat.data updates only those values without converting to dense.

2. Step 2: Save the updated sparse matrix back

   Using save_npz saves the modified sparse matrix efficiently.

3. Step 3: Check other options for correctness

   from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat = mat.toarray() + 5 save_npz('large_matrix.npz', mat) converts to dense explicitly, adds 5 to all elements including zeros, wastes memory, and save_npz fails on dense array. import numpy as np mat = np.load('large_matrix.npz') mat += 5 np.save('large_matrix.npz', mat) uses numpy load/save which does not handle sparse matrices. from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat.toarray() += 5 save_npz('large_matrix.npz', mat) tries to add 5 to a dense array but does not assign back, and wastes memory.

4. Final Answer:

   from scipy.sparse import load_npz, save_npz mat = load_npz('large_matrix.npz') mat.data += 5 save_npz('large_matrix.npz', mat) -> Option A

5. Quick Check:

   Modify mat.data for sparse update [OK]

Hint: Change mat.data to update non-zero sparse values [OK]

Common Mistakes:

• Adding scalar directly to sparse matrix (converts to dense)
• Using numpy load/save for sparse matrices
• Modifying dense array without saving changes