Eigenvalues and eigenvectors (eig) in MATLAB - Deep Dive

Overview - Eigenvalues and eigenvectors (eig)

What is it?

Eigenvalues and eigenvectors are special numbers and vectors associated with a square matrix. An eigenvector is a direction that does not change when the matrix acts on it, only its length changes. The amount it stretches or shrinks by is the eigenvalue. These concepts help us understand how matrices transform space.

Why it matters

Without eigenvalues and eigenvectors, we would struggle to analyze systems that change over time, like vibrations, population growth, or Google's search ranking. They simplify complex transformations into basic stretching along fixed directions, making problems easier to solve and understand.

Where it fits

Before learning eigenvalues and eigenvectors, you should know matrix multiplication and basic linear algebra concepts like vectors and matrices. After this, you can explore matrix diagonalization, principal component analysis (PCA), and solving differential equations.

Mental Model

Core Idea

Eigenvalues and eigenvectors reveal the fundamental directions and scales by which a matrix stretches or compresses space.

Think of it like...

Imagine a rubber sheet with arrows drawn on it. When you stretch or squeeze the sheet, most arrows change direction and length. But some arrows only get longer or shorter without changing direction. Those special arrows are like eigenvectors, and how much they stretch is the eigenvalue.

Matrix A acts on vector x:

  x (vector)  ---> [A] --->  y = A*x

If x is an eigenvector:

  A*x = λ*x

Where λ is the eigenvalue (a number).

Diagram:

  Vector x  --->  Matrix A  --->  Vector y (same direction as x, scaled by λ)

Build-Up - 7 Steps

1

Foundation

Understanding matrices and vectors

Concept: Learn what matrices and vectors are and how matrices multiply vectors.

A matrix is a grid of numbers arranged in rows and columns. A vector is a list of numbers, like a point or direction. Multiplying a matrix by a vector transforms the vector into a new vector. For example, in MATLAB: A = [2 0; 0 3]; x = [1; 1]; y = A*x; This multiplies matrix A by vector x to get y.

Result

y = [2; 3], which is the vector x stretched differently in each direction.

Understanding matrix-vector multiplication is essential because eigenvectors are special vectors that behave simply under this operation.

2

Foundation

Defining eigenvalues and eigenvectors

Concept: Introduce the formal definition of eigenvalues and eigenvectors.

An eigenvector x of a matrix A satisfies A*x = λ*x, where λ is a scalar called the eigenvalue. This means applying A to x only changes its length, not its direction. In MATLAB, eigenvalues and eigenvectors can be found using the eig function.

Result

Eigenvectors point in directions that remain unchanged by A, scaled by their eigenvalues.

Knowing this definition helps you identify the special vectors and numbers that reveal matrix behavior.

3

Intermediate

Computing eigenvalues and eigenvectors in MATLAB

🤔Before reading on: do you think MATLAB's eig function returns eigenvalues first or eigenvectors first? Commit to your answer.

Concept: Learn how to use MATLAB's eig function to find eigenvalues and eigenvectors.

In MATLAB, use [V,D] = eig(A); where A is a square matrix. - V is a matrix whose columns are eigenvectors. - D is a diagonal matrix with eigenvalues on the diagonal. Example: A = [4 1; 2 3]; [V,D] = eig(A); This finds eigenvalues and eigenvectors of A.

Result

D contains eigenvalues on its diagonal, V contains corresponding eigenvectors as columns.

Understanding the output format of eig helps you correctly interpret and use eigenvalues and eigenvectors in your analysis.

4

Intermediate

Interpreting eigenvalues and eigenvectors

🤔Before reading on: do you think eigenvectors are always unique? Commit to your answer.

Concept: Learn what eigenvalues and eigenvectors tell us about a matrix's effect.

Eigenvalues tell how much the eigenvector stretches or shrinks. If an eigenvalue is zero, the matrix squashes the vector to zero. If it's negative, the vector flips direction. Eigenvectors show directions that remain unchanged by the matrix transformation. Example: A = [0 1; -2 -3]; [V,D] = eig(A); Eigenvalues might be complex, indicating rotation.

Result

Eigenvalues and eigenvectors reveal stretching, shrinking, flipping, or rotating effects of the matrix.

Knowing how to interpret eigenvalues and eigenvectors helps you understand the nature of transformations in real problems.

5

Intermediate

Handling complex eigenvalues and eigenvectors

🤔Before reading on: do you think eigenvalues can be complex for real matrices? Commit to your answer.

Concept: Understand that eigenvalues and eigenvectors can be complex numbers, especially for certain matrices.

Some matrices, especially those representing rotations or oscillations, have complex eigenvalues and eigenvectors. MATLAB's eig function returns complex numbers when needed. Example: A = [0 -1; 1 0]; [V,D] = eig(A); This matrix represents a 90-degree rotation, eigenvalues are complex: i and -i.

Result

Complex eigenvalues indicate rotation or oscillation behavior in the system.

Recognizing complex eigenvalues expands your understanding of matrix transformations beyond simple stretching.

6

Advanced

Using eigen decomposition for matrix diagonalization

🤔Before reading on: do you think every matrix can be diagonalized using eigenvalues and eigenvectors? Commit to your answer.

Concept: Learn how eigenvalues and eigenvectors help rewrite a matrix in a simpler diagonal form.

If a matrix A has enough eigenvectors, it can be written as A = V*D*V^-1, where D is diagonal with eigenvalues, and V contains eigenvectors. This makes calculations like powers of A easier. Example: A = [4 1; 2 3]; [V,D] = eig(A); A_reconstructed = V*D*inv(V); This reconstructs A from its eigen decomposition.

Result

Diagonalization simplifies matrix operations and reveals structure.

Understanding diagonalization shows why eigenvalues and eigenvectors are powerful tools for simplifying complex matrix problems.

7

Expert

Numerical stability and eigenvalue computation challenges

🤔Before reading on: do you think MATLAB's eig function always gives perfectly accurate eigenvalues? Commit to your answer.

Concept: Explore the numerical challenges and limitations in computing eigenvalues and eigenvectors.

Computing eigenvalues numerically can be sensitive to rounding errors, especially for large or nearly defective matrices. MATLAB uses advanced algorithms like the QR algorithm to find eigenvalues, but small errors can occur. Example: A = [1 1e-10; 0 1]; [V,D] = eig(A); Eigenvalues are close but may differ slightly due to numerical precision.

Result

Numerical methods approximate eigenvalues; understanding limits prevents misinterpretation.

Knowing numerical stability issues helps you trust and verify eigenvalue computations in real-world data.

Under the Hood

MATLAB's eig function uses the QR algorithm, which iteratively decomposes the matrix into orthogonal and upper triangular matrices to converge on eigenvalues. Internally, it transforms the matrix into a simpler form (Hessenberg form) to speed up calculations. Eigenvectors are found by solving linear systems once eigenvalues are known.

Why designed this way?

The QR algorithm was chosen because it is efficient and numerically stable for a wide range of matrices. Earlier methods were slower or less stable. MATLAB's implementation balances speed and accuracy, making it suitable for practical use in engineering and science.

Matrix A
  │
  ▼
Hessenberg form (simplified matrix)
  │
  ▼ (QR iterations)
Converged to upper triangular matrix
  │
  ▼
Eigenvalues extracted from diagonal
  │
  ▼
Solve for eigenvectors

Myth Busters - 4 Common Misconceptions

Quick: do you think all matrices have real eigenvalues? Commit to yes or no.

Common Belief: All matrices have real eigenvalues.

Reality: Some matrices have complex eigenvalues, especially those representing rotations or oscillations.

Why it matters: Assuming eigenvalues are always real can lead to wrong conclusions about system behavior, like missing rotational dynamics.

Quick: do you think eigenvectors are always unique? Commit to yes or no.

Common Belief: Eigenvectors are always unique for each eigenvalue.

Reality: Eigenvectors are unique only up to scaling; any multiple of an eigenvector is also an eigenvector. Sometimes, multiple independent eigenvectors share the same eigenvalue (degeneracy).

Why it matters: Misunderstanding uniqueness can cause confusion when interpreting results or comparing eigenvectors.

Quick: do you think every matrix can be diagonalized? Commit to yes or no.

Common Belief: Every square matrix can be diagonalized using eigenvalues and eigenvectors.

Reality: Not all matrices are diagonalizable; some are defective and lack enough eigenvectors for diagonalization.

Why it matters: Expecting diagonalization always works can cause errors in algorithms relying on it, like certain matrix powers or decompositions.

Quick: do you think MATLAB's eig function always returns exact eigenvalues? Commit to yes or no.

Common Belief: MATLAB's eig function always returns exact eigenvalues.

Reality: eig returns numerical approximations that can have small errors due to floating-point arithmetic and matrix conditioning.

Why it matters: Ignoring numerical errors can lead to false confidence and mistakes in sensitive calculations.

Expert Zone

1

Eigenvectors can be chosen to be orthonormal if the matrix is symmetric, which simplifies many computations.

2

The order of eigenvalues and eigenvectors returned by eig is not guaranteed, so matching pairs requires care.

3

For large sparse matrices, specialized algorithms like Lanczos or Arnoldi methods are used instead of eig for efficiency.

When NOT to use

Avoid using eig for very large sparse matrices due to high computational cost; use iterative methods like eigs instead. Also, for non-square matrices, singular value decomposition (SVD) is the appropriate tool.

Production Patterns

In real-world systems, eigen decomposition is used for dimensionality reduction (PCA), stability analysis in control systems, vibration analysis in engineering, and Google's PageRank algorithm. Often, preprocessing ensures matrices are well-conditioned for reliable eigenvalue computation.

Connections

Principal Component Analysis (PCA)

Eigen decomposition is the mathematical foundation of PCA.

Understanding eigenvalues and eigenvectors helps grasp how PCA finds directions of maximum variance in data.

Quantum Mechanics

Eigenvalues and eigenvectors represent measurable quantities and states in quantum systems.

Knowing eigen concepts clarifies how physical properties emerge from mathematical operators in physics.

Vibrations and Modal Analysis (Mechanical Engineering)

Eigenvalues correspond to natural frequencies, eigenvectors to vibration modes.

Recognizing this link shows how math tools solve real engineering problems like building stability.

Common Pitfalls

#1 Confusing eigenvectors with any vector that changes under matrix multiplication.

Wrong approach:

A = [2 0; 0 3];
x = [1; 0];
y = A*x;
% Assuming x is eigenvector without checking if A*x = λ*x

Correct approach:

A = [2 0; 0 3];
[V,D] = eig(A);
% Verify if x matches any column in V

Root cause:

Misunderstanding the strict condition that eigenvectors must satisfy A*x = λ*x exactly.

#2 Using eig on non-square matrices expecting eigenvalues.

Wrong approach:

A = [1 2 3; 4 5 6];
[V,D] = eig(A); % This will error

Correct approach:

Use singular value decomposition instead:
[U,S,V] = svd(A);

Root cause:

Not knowing eigenvalues are defined only for square matrices.

#3 Assuming eigenvectors returned by eig are normalized.

Wrong approach:

[V,D] = eig(A);
% Using V directly assuming unit length

Correct approach:

Normalize eigenvectors explicitly:
V_norm = V ./ vecnorm(V);

Root cause:

Not realizing MATLAB does not guarantee eigenvectors have length 1.

Key Takeaways

Eigenvalues and eigenvectors reveal how matrices stretch or rotate space along special directions.

MATLAB's eig function computes these values and vectors, but understanding the output format is crucial.

Not all matrices have real or unique eigenvalues and eigenvectors; some have complex or repeated ones.

Numerical methods approximate eigenvalues, so small errors are normal and should be considered.

Eigen decomposition is foundational for many data science and engineering applications like PCA and system analysis.