Sparse iterative solvers (gmres, cg) in SciPy

Introduction

Sparse iterative solvers help solve big systems of equations quickly when most values are zero. They save time and memory.

When solving large linear systems from engineering or physics problems with many zero values.

When you want to find approximate solutions fast instead of exact ones.

When the matrix is too big to use direct methods like matrix inversion.

When working with sparse data in machine learning or network analysis.

When you want to save computer memory by avoiding dense matrix operations.

Syntax

SciPy

from scipy.sparse.linalg import gmres, cg

# gmres(A, b, tol=1e-5, maxiter=None)
# cg(A, b, tol=1e-5, maxiter=None)

# A: sparse matrix or linear operator
# b: right-hand side vector
# tol: tolerance for convergence
# maxiter: maximum iterations allowed

gmres works well for general matrices.

cg is faster but only works if the matrix is symmetric and positive definite.

Examples

Use gmres to solve Ax = b for a sparse matrix A.

SciPy

from scipy.sparse.linalg import gmres
import numpy as np
from scipy.sparse import diags

A = diags([2, 1, 3], [0, -1, 1], shape=(3,3))
b = np.array([1, 2, 3])
x, info = gmres(A, b)
print(x)

Use cg for symmetric positive definite sparse matrix A.

SciPy

from scipy.sparse.linalg import cg
import numpy as np
from scipy.sparse import diags

A = diags([4, 1, 1], [0, -1, 1], shape=(3,3))
b = np.array([1, 2, 3])
x, info = cg(A, b)
print(x)

Sample Program

This program creates a small sparse matrix and solves the system Ax = b using both gmres and cg solvers. It prints the solutions and info codes where 0 means the solver succeeded.

SciPy

from scipy.sparse.linalg import gmres, cg
import numpy as np
from scipy.sparse import diags

# Create a sparse matrix A
# Diagonal values: main diagonal 4, sub diagonal 1, super diagonal 1
A = diags([4, 1, 1], [0, -1, 1], shape=(3,3))

# Right-hand side vector b
b = np.array([1, 2, 3])

# Solve using gmres
x_gmres, info_gmres = gmres(A, b, tol=1e-8)

# Solve using cg
x_cg, info_cg = cg(A, b, tol=1e-8)

print("Solution with gmres:", x_gmres)
print("Info gmres (0 means success):", info_gmres)
print("Solution with cg:", x_cg)
print("Info cg (0 means success):", info_cg)

Important Notes

Check the info output: 0 means the solver found a solution successfully.

Set tol smaller for more accurate results but slower computation.

cg requires the matrix to be symmetric and positive definite; otherwise, it may fail.

Summary

Sparse iterative solvers like gmres and cg solve large sparse linear systems efficiently.

gmres works for general matrices; cg is faster but needs symmetric positive definite matrices.

They save memory and time by using the sparse structure instead of dense matrix methods.