Sparse iterative solvers (gmres, cg) in SciPy - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main purpose of sparse iterative solvers like GMRES and CG?

They efficiently solve large systems of linear equations where the matrix is sparse, meaning most elements are zero, saving memory and computation time.

beginner

What type of matrices is the Conjugate Gradient (CG) method best suited for?

CG is best for symmetric and positive definite matrices.

intermediate

How does GMRES differ from CG in terms of matrix requirements?

GMRES can solve general non-symmetric matrices, while CG requires the matrix to be symmetric and positive definite.

beginner

In SciPy, which module provides the gmres and cg functions?

The functions gmres and cg are available in the scipy.sparse.linalg module.

intermediate

Why are iterative solvers preferred over direct solvers for very large sparse systems?

Iterative solvers use less memory and can be faster because they avoid computing full matrix factorizations, which are costly for large sparse matrices.

Which solver is suitable for a symmetric positive definite sparse matrix?

A Conjugate Gradient (CG)

B GMRES

C LU decomposition

D Jacobi method

CG is designed for symmetric positive definite matrices, making it the best choice here.

What does GMRES stand for?

A Gaussian Matrix Residual

B Gradient Method for Residuals

C General Matrix Residual Solver

D Generalized Minimal Residual

GMRES stands for Generalized Minimal Residual, an iterative method for solving non-symmetric systems.

Which SciPy module contains the gmres and cg functions?

A scipy.linalg

B scipy.sparse.linalg

C scipy.optimize

D scipy.integrate

The gmres and cg functions are in scipy.sparse.linalg, specialized for sparse matrix operations.

Why are iterative solvers preferred for large sparse systems?

A They compute exact solutions instantly

B They require dense matrices

C They use less memory and can be faster

D They do not need initial guesses

Iterative solvers save memory and time by avoiding full matrix factorizations.

Which of these is NOT a characteristic of the CG method?

A Can solve non-symmetric matrices

B Works only on symmetric matrices

C Requires positive definiteness

D Is an iterative solver

CG cannot solve non-symmetric matrices; GMRES is used for those.

Explain when and why you would use the GMRES solver instead of CG.

Think about matrix symmetry and solver flexibility.

Describe the advantages of using sparse iterative solvers for large linear systems.

Consider memory and speed benefits.

Practice

1. Which of the following statements about scipy.sparse.linalg.cg is true?

easy

A. cg is slower than gmres for all matrices.

B. cg can solve any linear system regardless of matrix properties.

C. cg uses dense matrix methods internally.

D. cg requires the matrix to be symmetric and positive definite.

Solution

1. Step 1: Understand the requirements of cg

   The conjugate gradient method (cg) is designed for symmetric positive definite matrices only.

2. Step 2: Compare with other options

   cg cannot solve any matrix (B is wrong), it is often faster than gmres for suitable matrices (A is wrong), and it uses sparse methods, not dense (D is wrong).

3. Final Answer:

   cg requires the matrix to be symmetric and positive definite. -> Option D

4. Quick Check:

   cg needs symmetric positive definite matrix [OK]

Hint: Remember: CG needs symmetric positive definite matrices [OK]

Common Mistakes:

• Thinking CG works for any matrix
• Confusing CG with GMRES
• Assuming CG uses dense matrix methods

2. Which is the correct way to import the GMRES solver from SciPy?

easy

A. import scipy.linalg.gmres

B. from scipy.sparse.linalg import gmres

C. from scipy.linalg import gmres

D. import gmres from scipy.sparse

Solution

1. Step 1: Recall the module location of GMRES

   The GMRES solver is in scipy.sparse.linalg, so it must be imported from there.

2. Step 2: Check the import syntax

   Correct Python import syntax for a function is from module import function. from scipy.sparse.linalg import gmres matches this and the correct module.

3. Final Answer:

   from scipy.sparse.linalg import gmres -> Option B

4. Quick Check:

   Correct import syntax and module [OK]

Hint: Import gmres from scipy.sparse.linalg using 'from ... import' [OK]

Common Mistakes:

• Importing from scipy.linalg instead of sparse.linalg
• Using incorrect import syntax
• Trying to import gmres directly from scipy.sparse

3. What will be the output of the following code snippet?

import numpy as np
from scipy.sparse.linalg import cg
from scipy.sparse import diags

A = diags([1, 2, 1], [-1, 0, 1]).toarray()
b = np.array([4, 6, 4])
x, info = cg(A, b)
print(np.round(x, 2))

medium

A. [1. 2. 1.]

B. [2. 1. 2.]

C. [0. 0. 0.]

D. Error due to matrix not positive definite

Solution

1. Step 1: Analyze the matrix and vector

   The matrix A is tridiagonal with diagonals [1,2,1], which is symmetric positive definite. Vector b is [4,6,4].

2. Step 2: Solve using conjugate gradient

   Using cg, the solution x satisfies Ax = b. The solution is approximately [1, 2, 1].

3. Final Answer:

   [1. 2. 1.] -> Option A

4. Quick Check:

   cg solves Ax=b with symmetric positive definite A [OK]

Hint: Check matrix symmetry and positive definiteness before cg [OK]

Common Mistakes:

• Assuming cg fails on this matrix
• Confusing the solution vector with b
• Not rounding output before comparing

4. Identify the error in this code using cg solver:

import numpy as np
from scipy.sparse.linalg import cg

A = np.array([[0, 1], [1, 0]])
b = np.array([1, 2])
x, info = cg(A, b)
print(x)

medium

A. Matrix A is not symmetric positive definite, so cg will fail.

B. Vector b has wrong shape.

C. cg function is not imported correctly.

D. No error; code runs fine.

Solution

1. Step 1: Check matrix properties

   Matrix A = [[0,1],[1,0]] is symmetric but not positive definite (its eigenvalues are 1 and -1).

2. Step 2: Understand cg requirements

   The conjugate gradient method requires A to be symmetric positive definite. Since A is not, cg will fail or not converge properly.

3. Final Answer:

   Matrix A is not symmetric positive definite, so cg will fail. -> Option A

4. Quick Check:

   cg needs symmetric positive definite matrix [OK]

Hint: Check matrix eigenvalues before using cg [OK]

Common Mistakes:

• Ignoring matrix definiteness
• Assuming cg works for any symmetric matrix
• Thinking vector shape causes error

5. You have a large sparse matrix that is not symmetric positive definite. Which solver should you use to solve Ax = b efficiently?

hard

A. Use cg because it is always faster.

B. Convert matrix to dense and use direct solver.

C. Use gmres because it works for general matrices.

D. Use cg after transposing the matrix.

Solution

1. Step 1: Identify matrix properties

   The matrix is large, sparse, and not symmetric positive definite, so cg is not suitable.

2. Step 2: Choose appropriate solver

   gmres can handle general matrices efficiently without requiring symmetry or positive definiteness.

3. Step 3: Avoid dense conversion

   Converting to dense wastes memory and time, so it is not efficient.

4. Final Answer:

   Use gmres because it works for general matrices. -> Option C

5. Quick Check:

   gmres handles general sparse matrices [OK]

Hint: Use gmres for non-symmetric or indefinite sparse matrices [OK]

Common Mistakes:

• Trying to use cg on non-symmetric matrices
• Converting sparse to dense unnecessarily
• Thinking transposing fixes definiteness