CSR format (Compressed Sparse Row) in SciPy - Mini Project: Build & Apply

Working with CSR format (Compressed Sparse Row) using scipy

📖 Scenario: Imagine you have a large table of numbers mostly filled with zeros. This is common in real-world data like user ratings or connections in social networks. Storing all those zeros wastes space. The CSR format helps us store only the important numbers efficiently.

🎯 Goal: You will create a sparse matrix using CSR format with scipy, configure a threshold to filter values, extract the filtered data, and finally print the filtered matrix data.

📋 What You'll Learn

Create a sparse matrix in CSR format using scipy

Define a threshold value to filter matrix data

Use CSR matrix attributes to filter data based on the threshold

Print the filtered data arrays

💡 Why This Matters

🌍 Real World

Sparse matrices are used in recommendation systems, natural language processing, and network analysis where data is mostly zeros but some values are important.

💼 Career

Understanding CSR format helps in efficiently storing and processing large datasets in data science and machine learning jobs.

1

Create a CSR sparse matrix

Import csr_matrix from scipy.sparse and create a CSR matrix called matrix from this 3x4 dense array: [[0, 0, 1, 0], [2, 0, 0, 3], [0, 4, 0, 0]]

SciPy

# Your code here

Hint

Use csr_matrix() to convert a list of lists into a sparse matrix.

2

Set a threshold value

Create a variable called threshold and set it to 2 to filter matrix values greater than this number.

SciPy

from scipy.sparse import csr_matrix

matrix = csr_matrix([[0, 0, 1, 0], [2, 0, 0, 3], [0, 4, 0, 0]])
# Your code here

Hint

Just assign the number 2 to a variable named threshold.

3

Filter matrix data using the threshold

Create a new variable called filtered_data that contains only the values from matrix.data which are greater than threshold using a list comprehension.

SciPy

from scipy.sparse import csr_matrix

matrix = csr_matrix([[0, 0, 1, 0], [2, 0, 0, 3], [0, 4, 0, 0]])

threshold = 2
# Your code here

Hint

Use a list comprehension to pick values from matrix.data that are greater than threshold.

4

Print the filtered data

Print the variable filtered_data to show the filtered values.

SciPy

from scipy.sparse import csr_matrix

matrix = csr_matrix([[0, 0, 1, 0], [2, 0, 0, 3], [0, 4, 0, 0]])

threshold = 2

filtered_data = [value for value in matrix.data if value > threshold]
# Your code here

Hint

Use print(filtered_data) to display the filtered values.

Practice

1. What is the main advantage of using the CSR format for storing matrices in data science?

easy

A. It stores all elements including zeros for faster access.

B. It saves memory by storing only non-zero elements and their positions.

C. It converts matrices into a list of lists format.

D. It automatically normalizes the matrix values.

Solution

1. Step 1: Understand CSR format purpose

   CSR format is designed to save space by storing only non-zero values and their positions in the matrix.

2. Step 2: Compare options with CSR features

   It saves memory by storing only non-zero elements and their positions. correctly describes this space-saving feature. Other options describe unrelated or incorrect features.

3. Final Answer:

   It saves memory by storing only non-zero elements and their positions. -> Option B

4. Quick Check:

   CSR = stores non-zero only [OK]

Hint: Remember CSR stores only non-zero values and their indices [OK]

Common Mistakes:

• Thinking CSR stores all elements including zeros
• Confusing CSR with dense matrix formats
• Assuming CSR normalizes data automatically

2. Which of the following is the correct way to create a CSR matrix from a dense NumPy array arr using SciPy?

easy

A. csr_matrix.to_csr(arr)

B. csr(arr)

C. sparse.csr(arr)

D. csr_matrix(arr)

Solution

1. Step 1: Recall SciPy CSR matrix creation syntax

   The correct function to create a CSR matrix from a dense array is csr_matrix() from scipy.sparse.

2. Step 2: Check each option

   csr_matrix(arr) uses the correct function name and syntax. Options B, C, and D are incorrect or invalid function calls.

3. Final Answer:

   csr_matrix(arr) -> Option D

4. Quick Check:

   Use csr_matrix() to convert dense to CSR [OK]

Hint: Use csr_matrix() function from scipy.sparse [OK]

Common Mistakes:

• Using incorrect function names like csr() or sparse.csr()
• Trying to call to_csr() method which doesn't exist
• Confusing with other sparse formats like COO

3. Given the following code, what will be the output of print(csr.data)?

import numpy as np
from scipy.sparse import csr_matrix

arr = np.array([[0, 0, 1], [2, 0, 0], [0, 3, 0]])
csr = csr_matrix(arr)
print(csr.data)

medium

A. [1 2 3]

B. [0 0 1 2 0 0 0 3 0]

C. [1 2 0 3]

D. [1 2 3 0]

Solution

1. Step 1: Understand csr.data attribute

   The data attribute of a CSR matrix stores only the non-zero values in row-major order.

2. Step 2: Identify non-zero elements in arr

   The non-zero elements are 1 (row 0, col 2), 2 (row 1, col 0), and 3 (row 2, col 1). So csr.data will be [1, 2, 3].

3. Final Answer:

   [1 2 3] -> Option A

4. Quick Check:

   csr.data = non-zero values list [OK]

Hint: csr.data holds only non-zero values in order [OK]

Common Mistakes:

• Expecting csr.data to include zeros
• Confusing csr.data with the full dense array
• Misordering the non-zero elements

4. Identify the error in the following code snippet that attempts to create a CSR matrix:

from scipy.sparse import csr_matrix

arr = [[0, 1], [2, 0]]
csr = csr_matrix(arr, shape=(2, 1))
print(csr.shape)

medium

A. The shape parameter is incompatible with the input array size.

B. The input array must be a NumPy array, not a list.

C. csr_matrix does not accept the shape parameter.

D. The print statement syntax is incorrect.

Solution

1. Step 1: Check input array and shape parameter

   The input array has shape (2, 2), but the shape parameter is set to (2, 1), which conflicts.

2. Step 2: Understand csr_matrix shape rules

   The shape parameter must match or be compatible with the input data shape. Here, (2, 1) is incompatible with (2, 2) input.

3. Final Answer:

   The shape parameter is incompatible with the input array size. -> Option A

4. Quick Check:

   Shape must match input array size [OK]

Hint: Ensure shape matches input array dimensions [OK]

Common Mistakes:

• Assuming lists cannot be input to csr_matrix
• Thinking csr_matrix disallows shape parameter
• Misreading print statement as error

5. You have a large sparse matrix representing user-item ratings with many zeros. You want to efficiently compute the sum of all ratings for each user (row). Which approach using CSR format is best?

hard

A. Use the CSR matrix's sum(axis=0) method to sum rows.

B. Convert the CSR matrix back to a dense array and use NumPy sum along axis 1.

C. Use the CSR matrix's sum(axis=1) method to sum non-zero values per row efficiently.

D. Manually iterate over all elements including zeros to sum each row.

Solution

1. Step 1: Understand CSR matrix row sum

   CSR format stores data row-wise, so summing along axis 1 is efficient and skips zeros.

2. Step 2: Evaluate options for efficiency

   Use the CSR matrix's sum(axis=1) method to sum non-zero values per row efficiently. uses sum(axis=1) directly on CSR, which is fast and memory efficient. Convert the CSR matrix back to a dense array and use NumPy sum along axis 1. wastes memory converting to dense. Manually iterate over all elements including zeros to sum each row. is slow. Use the CSR matrix's sum(axis=0) method to sum rows. sums columns, not rows.

3. Final Answer:

   Use the CSR matrix's sum(axis=1) method to sum non-zero values per row efficiently. -> Option C

4. Quick Check:

   Sum rows with csr.sum(axis=1) [OK]

Hint: Sum rows with csr.sum(axis=1) for efficiency [OK]

Common Mistakes:

• Converting to dense wastes memory
• Summing axis=0 sums columns, not rows
• Iterating manually is slow and unnecessary