CSC format (Compressed Sparse Column) in SciPy - Step-by-Step Execution

Concept Flow - CSC format (Compressed Sparse Column)

Start with dense matrix

↓

Identify non-zero elements

↓

Store values in 'data' array

↓

Store row indices of these values

↓

Store column pointers in 'indptr'

↓

Create CSC matrix object

↓

Use for efficient column slicing and operations

CSC format stores a sparse matrix by columns: values, row indices, and column pointers for fast column access.

Execution Sample

SciPy

import numpy as np
from scipy.sparse import csc_matrix

A = np.array([[0,0,1],[2,0,0],[0,3,0]])
csc = csc_matrix(A)
print(csc.data, csc.indices, csc.indptr)

Convert a small dense matrix to CSC format and print its internal arrays.

Execution Table

Step	Action	Values (data)	Row Indices (indices)	Column Pointers (indptr)
1	Start with matrix A	-	-	-
2	Find non-zero in col 0	[2]	[1]	[0]
3	Find non-zero in col 1	[2,3]	[1,2]	[0,1]
4	Find non-zero in col 2	[2,3,1]	[1,2,0]	[0,1,2]
5	Add end pointer for last col	[2,3,1]	[1,2,0]	[0,1,2,3]
6	Create CSC matrix with these arrays	[2,3,1]	[1,2,0]	[0,1,2,3]

💡 All columns processed, CSC arrays fully built.

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	Final
data	[]	[2]	[2,3]	[2,3,1]	[2,3,1]	[2,3,1]
indices	[]	[1]	[1,2]	[1,2,0]	[1,2,0]	[1,2,0]
indptr	[0]	[0]	[0,1]	[0,1,2]	[0,1,2,3]	[0,1,2,3]

Key Moments - 3 Insights

Why does 'indptr' have one more element than the number of columns?

'indptr' marks start of each column in 'data' and ends with total non-zero count, so it has length columns+1 (see execution_table step 5).

Why are row indices not sorted within each column?

Row indices correspond to order of non-zero elements found top to bottom in each column, preserving original matrix order (see execution_table steps 2-4).

What does 'data' array represent exactly?

'data' stores all non-zero values column by column, matching 'indices' for row positions (see execution_table steps 2-4).

Visual Quiz - 3 Questions

Look at the execution_table at step 4, what is the content of 'indices'?

A [1,2,0]

B [0,1,2]

C [2,3,1]

D [0,0,0]

💡 Hint

Check the 'Row Indices (indices)' column at step 4 in execution_table.

At which step does 'indptr' get its final length of 4?

A Step 3

B Step 5

C Step 4

D Step 6

💡 Hint

Look at 'Column Pointers (indptr)' column in execution_table to see when length changes to 4.

If the matrix had an extra zero column at the end, how would 'indptr' change?

A It would have one less element

B It would have same length but different values

C It would have one more element with same last value repeated

D It would be empty

💡 Hint

Recall 'indptr' length is columns+1 and last value equals total non-zero count.

Concept Snapshot

CSC format stores sparse matrices by columns.
It uses three arrays: data (non-zero values), indices (row numbers), and indptr (column start pointers).
'indptr' length is number of columns + 1.
Efficient for column slicing and matrix operations.
Used in scipy.sparse.csc_matrix.

Full Transcript

CSC format (Compressed Sparse Column) stores a sparse matrix by columns. It keeps three arrays: 'data' for non-zero values, 'indices' for their row positions, and 'indptr' which points to where each column starts in 'data'. The 'indptr' array has one more element than the number of columns, marking the start of each column and the end of the last. This format is efficient for column-based operations. The example code converts a small dense matrix to CSC and prints these arrays. Step by step, non-zero values and their row indices are collected column by column, and column pointers are built. This helps understand how CSC stores sparse data internally.

Practice

1. What does the CSC format in scipy.sparse primarily store for a sparse matrix?

easy

A. Only non-zero values and their row indices, organized by columns

B. All values including zeros, organized by rows

C. Only zero values and their column indices

D. All values including zeros, organized by columns

Solution

1. Step 1: Understand CSC format storage

   CSC stands for Compressed Sparse Column, which stores only non-zero values and their row indices, grouped by columns.

2. Step 2: Compare options with CSC definition

   Only non-zero values and their row indices, organized by columns matches the CSC format definition exactly, while others include zeros or wrong organization.

3. Final Answer:

   Only non-zero values and their row indices, organized by columns -> Option A

4. Quick Check:

   CSC stores non-zero values by columns = A [OK]

Hint: CSC stores non-zero values by columns only [OK]

Common Mistakes:

• Thinking CSC stores zero values
• Confusing CSC with CSR (row-based)
• Assuming all values are stored

2. Which of the following is the correct way to create a CSC matrix from a dense NumPy array arr in Python using SciPy?

easy

A. scipy.sparse.csc_matrix(arr)

B. scipy.sparse.csr_matrix(arr)

C. scipy.sparse.dense_matrix(arr)

D. scipy.sparse.csc(arr)

Solution

1. Step 1: Identify correct SciPy function for CSC

   The function to create a CSC matrix is scipy.sparse.csc_matrix().

2. Step 2: Check options for syntax correctness

   scipy.sparse.csc_matrix(arr) uses the correct function and syntax. scipy.sparse.csr_matrix(arr) creates CSR, not CSC. Options C and D are invalid functions.

3. Final Answer:

   scipy.sparse.csc_matrix(arr) -> Option A

4. Quick Check:

   Use csc_matrix() to create CSC format [OK]

Hint: Use csc_matrix() to create CSC sparse matrix [OK]

Common Mistakes:

• Using csr_matrix instead of csc_matrix
• Trying non-existent functions like dense_matrix
• Missing _matrix suffix in function name

3. Given the following code, what will be the output of print(csc_matrix.indptr)?

import numpy as np
from scipy.sparse import csc_matrix

arr = np.array([[0, 0, 3],
                [4, 0, 0],
                [0, 5, 6]])
csc_matrix = csc_matrix(arr)

medium

A. [0 2 3 4]

B. [0 1 2 3]

C. [0 1 3 4]

D. [0 1 2 4]

Solution

1. Step 1: Understand indptr in CSC format

   indptr array shows column start indices in data arrays. For 3 columns, length is 4.

2. Step 2: Count non-zero elements per column

   Column 0 has 1 non-zero (4), column 1 has 1 non-zero (5), column 2 has 2 non-zeros (3,6). So indptr = [0,1,2,4].

3. Final Answer:

   [0 1 2 4] -> Option D

4. Quick Check:

   indptr counts cumulative non-zeros per column = B [OK]

Hint: Count non-zeros per column cumulatively for indptr [OK]

Common Mistakes:

• Counting rows instead of columns
• Mixing up indptr with indices array
• Off-by-one errors in cumulative counts

4. Identify the error in this code snippet that tries to create a CSC matrix:

from scipy.sparse import csc_matrix

arr = [[1, 0], [0, 2]]
csc = csc_matrix(arr, shape=(3,3))
print(csc.toarray())

medium

A. toarray() method does not exist for csc_matrix

B. Input array must be a NumPy array, not a list

C. Shape parameter is larger than input array size causing mismatch

D. csc_matrix requires data type float, not int

Solution

1. Step 1: Check input array and shape parameter

   Input array is 2x2 but shape is set to 3x3, causing dimension mismatch.

2. Step 2: Understand effect of shape mismatch

   Specifying a larger shape than data size is invalid and raises an error.

3. Final Answer:

   Shape parameter is larger than input array size causing mismatch -> Option C

4. Quick Check:

   Shape must match input size or smaller [OK]

Hint: Shape must match or be smaller than input array size [OK]

Common Mistakes:

• Assuming lists are invalid input
• Thinking data type must be float
• Believing toarray() is missing

5. You have a large sparse matrix representing user-item ratings with mostly zeros. You want to efficiently compute the sum of ratings for each item (column). Which approach using CSC format is best?

hard

A. Convert to dense array and sum all values using np.sum()

B. Convert to CSC format and sum non-zero values column-wise using .sum(axis=0)

C. Convert to CSR format and sum non-zero values row-wise using .sum(axis=1)

D. Use COO format and iterate over all entries to sum columns manually

Solution

1. Step 1: Understand CSC format advantage

   CSC stores data column-wise, making column operations like sum efficient.

2. Step 2: Choose best method for column sums

   Using .sum(axis=0) on CSC matrix efficiently sums columns without converting to dense.

3. Final Answer:

   Convert to CSC format and sum non-zero values column-wise using .sum(axis=0) -> Option B

4. Quick Check:

   CSC + sum(axis=0) = fast column sums [OK]

Hint: Use CSC and sum(axis=0) for fast column sums [OK]

Common Mistakes:

• Using CSR which is row-based for column sums
• Converting to dense wastes memory
• Manually iterating instead of vectorized sum