Eigenvalue problems (eigs, eigsh) in SciPy - Step-by-Step Execution

Concept Flow - Eigenvalue problems (eigs, eigsh)

Start: Define matrix A

↓

Choose eigs or eigsh

↓

Call eigs/eigsh with A

↓

Compute eigenvalues and eigenvectors

↓

Return eigenvalues, eigenvectors

↓

Use results for analysis or visualization

We start with a matrix, pick the right solver (eigs for general, eigsh for symmetric), compute eigenvalues and eigenvectors, then use them.

Execution Sample

SciPy

import numpy as np
from scipy.sparse.linalg import eigsh
A = np.array([[2, 1], [1, 2]])
vals, vecs = eigsh(A, k=1)
print(vals, vecs)

This code finds the largest eigenvalue and eigenvector of matrix A using eigsh.

Execution Table

Step	Action	Matrix A	k (number of eigenvalues)	Eigenvalues computed	Eigenvectors computed
1	Define matrix A	[[2 1] [1 2]]	N/A	N/A	N/A
2	Choose eigsh solver	[[2 1] [1 2]]	1	N/A	N/A
3	Call eigsh(A, k=1)	[[2 1] [1 2]]	1	N/A	N/A
4	Compute eigenvalues and eigenvectors	[[2 1] [1 2]]	1	[3.0]	[[0.70710678] [0.70710678]]
5	Return results	[[2 1] [1 2]]	1	[3.0]	[[0.70710678] [0.70710678]]

💡 Eigenvalues and eigenvectors computed for k=1, computation ends.

Variable Tracker

Variable	Start	After Step 1	After Step 4	Final
A	undefined	[[2 1] [1 2]]	[[2 1] [1 2]]	[[2 1] [1 2]]
k	undefined	undefined	1	1
vals	undefined	undefined	[3.0]	[3.0]
vecs	undefined	undefined	[[0.70710678] [0.70710678]]	[[0.70710678] [0.70710678]]

Key Moments - 3 Insights

Why do we use eigsh instead of eigs for symmetric matrices?

eigsh is optimized for symmetric or Hermitian matrices and is faster and more accurate, as shown in step 2 where we choose the solver based on matrix properties.

What does the parameter k mean in eigs/eigsh?

k is the number of eigenvalues and eigenvectors to compute, as seen in step 3 and 4 where k=1 means we get the largest eigenvalue and its vector.

Why are eigenvalues complex numbers even for real symmetric matrices?

eigs returns complex numbers by default for general matrices; for symmetric matrices, eigenvalues are real and returned as real numbers by eigsh, like 3.0 in step 4.

Visual Quiz - 3 Questions

Look at the execution table at step 4, what is the eigenvalue computed?

A [3.0]

B [2.0]

C [1.0]

D [0.0]

💡 Hint

Check the 'Eigenvalues computed' column at step 4 in the execution_table.

At which step does the code actually compute eigenvalues and eigenvectors?

A Step 2

B Step 3

C Step 4

D Step 5

💡 Hint

Look for the step where 'Compute eigenvalues and eigenvectors' happens in the execution_table.

If we change k from 1 to 2, how would the eigenvalues in the table change?

A Eigenvalues remain the same

B We get two eigenvalues instead of one

C No eigenvalues are computed

D Eigenvalues become zero

💡 Hint

Refer to the meaning of k in the key_moments and execution_table steps 3 and 4.

Concept Snapshot

Use scipy.sparse.linalg.eigs or eigsh to find eigenvalues and eigenvectors.
- eigs: for general square matrices.
- eigsh: for symmetric/Hermitian matrices (faster, more accurate).
- Parameter k controls how many eigenvalues/vectors to compute.
- Returns complex eigenvalues and eigenvectors (even if imaginary part is zero) for eigs; eigsh returns real values for symmetric matrices.
- Useful for large sparse matrices.

Full Transcript

We start by defining a matrix A. Then we choose the solver: eigs for general matrices or eigsh for symmetric ones. We call the solver with A and the number k of eigenvalues to find. The solver computes eigenvalues and eigenvectors and returns them. For example, with A = [[2,1],[1,2]] and k=1, eigsh returns the largest eigenvalue 3.0 and its eigenvector. The eigenvalues are real numbers for symmetric matrices. The parameter k controls how many eigenvalues and eigenvectors we get. Using eigsh for symmetric matrices is faster and more accurate. This process helps us analyze matrix properties in data science and engineering.

Practice

1. What is the main difference between scipy.sparse.linalg.eigs and scipy.sparse.linalg.eigsh?

easy

A. eigs returns eigenvectors only, while eigsh returns eigenvalues only.

B. eigs works for any square matrix, while eigsh is optimized for symmetric or Hermitian matrices.

C. eigsh works for any square matrix, while eigs only works for diagonal matrices.

D. eigsh is used for non-square matrices, while eigs is for square matrices.

Solution

1. Step 1: Understand the function purposes

   eigs is designed to find eigenvalues and eigenvectors of any square matrix, including non-symmetric ones. eigsh is a specialized version optimized for symmetric or Hermitian matrices, which are common in many applications.

2. Step 2: Compare matrix types each function supports

   eigsh takes advantage of symmetry to be faster and more accurate, but it requires the matrix to be symmetric. eigs has no such restriction but may be slower.

3. Final Answer:

   eigs works for any square matrix, while eigsh is optimized for symmetric or Hermitian matrices. -> Option B

4. Quick Check:

   Function specialization = C [OK]

Hint: Remember: eigsh = symmetric only, eigs = any square matrix [OK]

Common Mistakes:

• Thinking eigsh works for any matrix
• Confusing eigs and eigsh outputs
• Assuming eigsh works for non-square matrices

2. Which of the following is the correct way to import and use eigsh from scipy.sparse.linalg to compute 3 eigenvalues of a symmetric matrix A?

easy

A. from scipy.sparse.linalg import eigsh

vals, vecs = eigsh(A, k=3)

B. import scipy.linalg as la

vals, vecs = la.eigsh(A, 3)

C. from scipy.linalg import eigsh

vals, vecs = eigsh(A, 3)

D. from scipy.sparse.linalg import eigs

vals, vecs = eigs(A, k=3)

Solution

1. Step 1: Check the correct import statement

   eigsh is in scipy.sparse.linalg, so the import must be from there, not scipy.linalg.

2. Step 2: Verify function call syntax

   The function call requires the matrix A and the number of eigenvalues k=3. from scipy.sparse.linalg import eigsh vals, vecs = eigsh(A, k=3) uses correct syntax and import.

3. Final Answer:

   from scipy.sparse.linalg import eigsh vals, vecs = eigsh(A, k=3) -> Option A

4. Quick Check:

   Correct import and call = D [OK]

Hint: Import eigsh from scipy.sparse.linalg and use k=number [OK]

Common Mistakes:

• Importing eigsh from scipy.linalg instead of scipy.sparse.linalg
• Using eigs instead of eigsh for symmetric matrices
• Passing number without keyword k

3. Given the code below, what will be the output of print(vals)?

import numpy as np
from scipy.sparse.linalg import eigsh

A = np.array([[2, 1], [1, 2]])
vals, vecs = eigsh(A, k=1, which='LM')
print(np.round(vals, 2))

medium

A. [2.00]

B. [1.00]

C. [3.00]

D. [0.00]

Solution

1. Step 1: Understand the matrix and eigenvalues

   Matrix A is symmetric with values [[2,1],[1,2]]. Its eigenvalues are 3 and 1.

2. Step 2: Check the function call parameters

   eigsh is called with k=1 and which='LM' meaning largest magnitude eigenvalue. So it returns the largest eigenvalue, which is 3.

3. Final Answer:

   [3.00] -> Option C

4. Quick Check:

   Largest eigenvalue = 3.00 [OK]

Hint: which='LM' returns largest eigenvalue [OK]

Common Mistakes:

• Confusing largest eigenvalue with smallest
• Not rounding output
• Using eigs instead of eigsh for symmetric matrix

4. The following code raises an error. What is the most likely cause?

import numpy as np
from scipy.sparse.linalg import eigsh

A = np.array([[1, 2], [3, 4]])
vals, vecs = eigsh(A, k=1)

medium

A. The import statement is incorrect.

B. The value of k is too large for the matrix size.

C. eigsh requires the matrix to be sparse, but A is dense.

D. Matrix A is not symmetric, so eigsh cannot be used.

Solution

1. Step 1: Check matrix properties

   Matrix A = [[1,2],[3,4]] is not symmetric because A[0,1] != A[1,0].

2. Step 2: Understand eigsh requirements

   eigsh requires the matrix to be symmetric or Hermitian. Using it on a non-symmetric matrix causes an error.

3. Final Answer:

   Matrix A is not symmetric, so eigsh cannot be used. -> Option D

4. Quick Check:

   Symmetry required for eigsh = A [OK]

Hint: Check matrix symmetry before using eigsh [OK]

Common Mistakes:

• Assuming eigsh works on any matrix
• Thinking k=1 is too large for 2x2 matrix
• Believing eigsh only works on sparse matrices

5. You have a large symmetric matrix representing connections in a social network. You want to find the 5 smallest eigenvalues to analyze community structure. Which approach is best?

hard

A. Use eigsh with k=5 and which='SM' to get the smallest eigenvalues efficiently.

B. Use eigs with k=5 and which='LM' to get the largest eigenvalues.

C. Convert the matrix to dense and use numpy.linalg.eig to get all eigenvalues.

D. Use eigsh with k=5 and which='LM' to get the largest eigenvalues.

Solution

1. Step 1: Identify matrix type and goal

   The matrix is large and symmetric, and we want the 5 smallest eigenvalues to study community structure.

2. Step 2: Choose appropriate function and parameters

   eigsh is optimized for symmetric matrices. Using k=5 and which='SM' returns the smallest magnitude eigenvalues efficiently without computing all eigenvalues.

3. Step 3: Evaluate other options

   eigs is less efficient for symmetric matrices. Converting to dense is costly for large matrices. Getting largest eigenvalues is not the goal.

4. Final Answer:

   Use eigsh with k=5 and which='SM' to get the smallest eigenvalues efficiently. -> Option A

5. Quick Check:

   Symmetric + smallest eigenvalues = eigsh + which='SM' [OK]

Hint: For smallest eigenvalues of symmetric matrix, use eigsh with which='SM' [OK]

Common Mistakes:

• Using eigs instead of eigsh for symmetric matrix
• Requesting largest eigenvalues instead of smallest
• Converting large sparse matrix to dense unnecessarily