Principal Component Analysis (PCA) in MATLAB - Step-by-Step Execution

Concept Flow - Principal Component Analysis (PCA)

Start with data matrix X

↓

Center data by subtracting mean

↓

Calculate covariance matrix

↓

Compute eigenvalues and eigenvectors

↓

Sort eigenvectors by eigenvalues descending

↓

Project data onto top eigenvectors (principal components)

↓

Get transformed data with reduced dimensions

PCA transforms data by centering it, finding directions of maximum variance (eigenvectors), and projecting data onto these directions to reduce dimensions.

Execution Sample

MATLAB

X = [2 0; 0 1; 1 3];
X_centered = X - mean(X);
C = cov(X_centered);
[V,D] = eig(C);
[~,idx] = sort(diag(D),'descend');
V = V(:,idx);
X_pca = X_centered * V;

This code centers data, computes covariance, finds eigenvectors, sorts them, and projects data onto principal components.

Execution Table

Step	Action	Variable	Value/Result
1	Input data matrix X	X	[2 0; 0 1; 1 3]
2	Calculate mean of X	mean(X)	[1 1.3333]
3	Center data by subtracting mean	X_centered	[1 -1.3333; -1 -0.3333; 0 1.6667]
4	Calculate covariance matrix	C	[1 1.5; 1.5 2.3333]
5	Compute eigenvalues and eigenvectors	D, V	D=diag([0.0419, 3.2914]), V=[-0.8321 0.5546; -0.5546 -0.8321]
6	Sort eigenvalues descending and reorder eigenvectors	idx, V	idx=[2 1], V=[0.5546 -0.8321; -0.8321 -0.5546]
7	Project centered data onto eigenvectors	X_pca	[1.1102 -0.6667; -0.4999 -0.8333; -0.6103 1.5]
8	End	Result	X_pca is data in principal component space

💡 Projection complete; data transformed into principal component space

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	After Step 7	Final
X	[2 0;0 1;1 3]	[2 0;0 1;1 3]	[2 0;0 1;1 3]	[2 0;0 1;1 3]	[2 0;0 1;1 3]	[2 0;0 1;1 3]	[2 0;0 1;1 3]	[2 0;0 1;1 3]
mean(X)	N/A	[1 1.3333]	[1 1.3333]	[1 1.3333]	[1 1.3333]	[1 1.3333]	[1 1.3333]	[1 1.3333]
X_centered	N/A	N/A	[1 -1.3333; -1 -0.3333; 0 1.6667]	[1 -1.3333; -1 -0.3333; 0 1.6667]	[1 -1.3333; -1 -0.3333; 0 1.6667]	[1 -1.3333; -1 -0.3333; 0 1.6667]	[1 -1.3333; -1 -0.3333; 0 1.6667]	[1 -1.3333; -1 -0.3333; 0 1.6667]
C	N/A	N/A	N/A	[1 1.5; 1.5 2.3333]	[1 1.5; 1.5 2.3333]	[1 1.5; 1.5 2.3333]	[1 1.5; 1.5 2.3333]	[1 1.5; 1.5 2.3333]
D	N/A	N/A	N/A	N/A	[0.0419, 3.2914]	[0.0419, 3.2914]	[0.0419, 3.2914]	[0.0419, 3.2914]
V	N/A	N/A	N/A	N/A	[ -0.8321 0.5546; -0.5546 -0.8321]	[0.5546 -0.8321; -0.8321 -0.5546]	[0.5546 -0.8321; -0.8321 -0.5546]	[0.5546 -0.8321; -0.8321 -0.5546]
X_pca	N/A	N/A	N/A	N/A	N/A	N/A	[1.1102 -0.6667; -0.4999 -0.8333; -0.6103 1.5]	[1.1102 -0.6667; -0.4999 -0.8333; -0.6103 1.5]

Key Moments - 3 Insights

Why do we subtract the mean from the data before calculating covariance?

Subtracting the mean centers the data around zero, ensuring covariance measures variance relative to the average. See step 3 in execution_table where X_centered is calculated.

Why do we sort eigenvalues and eigenvectors before projecting data?

Sorting ensures we use principal components with the highest variance first, capturing most information. Refer to step 6 where eigenvalues and eigenvectors are sorted.

What does projecting data onto eigenvectors achieve?

It transforms data into a new space where axes represent directions of maximum variance, reducing dimensions if needed. See step 7 where X_pca is computed.

Visual Quiz - 3 Questions

Look at the execution_table at step 3. What is the value of X_centered?

A [2 0; 0 1; 1 3]

B [1 -1.3333; -1 -0.3333; 0 1.6667]

C [0 0; 0 0; 0 0]

D [1 1; 1 1; 1 1]

💡 Hint

Check the 'Value/Result' column at step 3 in execution_table.

At which step does the condition 'sort eigenvalues descending' happen?

A Step 4

B Step 5

C Step 6

D Step 7

💡 Hint

Look for sorting action in execution_table under 'Action' column.

If we skip centering data (step 3), how would X_pca values change?

A They would be shifted and not represent principal components correctly

B They would be zeros

C They would be the same as shown

D They would be negative of current values

💡 Hint

Refer to variable_tracker for X_centered and understand its role before projection.

Concept Snapshot

PCA steps:
1. Center data by subtracting mean.
2. Compute covariance matrix.
3. Find eigenvalues and eigenvectors.
4. Sort eigenvectors by eigenvalues descending.
5. Project data onto top eigenvectors.
Result: data in reduced dimension space capturing most variance.

Full Transcript

Principal Component Analysis (PCA) starts with a data matrix X. We first calculate the mean of each column and subtract it from the data to center it around zero. This centered data is used to compute the covariance matrix, which measures how variables vary together. Next, we find eigenvalues and eigenvectors of the covariance matrix. Eigenvectors represent directions of variance, and eigenvalues tell us how much variance is in each direction. We sort eigenvectors by descending eigenvalues to prioritize directions with most variance. Finally, we project the centered data onto these eigenvectors to get transformed data in principal component space. This reduces dimensions while keeping important information. The execution table shows each step with variable values, and the variable tracker follows how variables change. Key moments clarify why centering and sorting are important. The visual quiz tests understanding of these steps.

Practice

1.

What is the main purpose of Principal Component Analysis (PCA) in MATLAB?

easy

A. To reduce the number of variables by finding main directions of data

B. To sort data in ascending order

C. To calculate the sum of all data points

D. To generate random numbers for data simulation

Solution

1. Step 1: Understand PCA goal

   PCA finds new axes (principal components) that capture most data variation.

2. Step 2: Identify main purpose

   It reduces data dimensions by focusing on important directions, simplifying analysis.

3. Final Answer:

   To reduce the number of variables by finding main directions of data -> Option A

4. Quick Check:

   PCA = Dimensionality reduction [OK]

Hint: PCA simplifies data by finding main directions [OK]

Common Mistakes:

• Thinking PCA sorts data
• Confusing PCA with summing data
• Assuming PCA generates random data

2.

Which MATLAB command correctly performs PCA on a data matrix X?

easy

A. score = pca(X, 'score');

B. pca = X.coeff();

C. [coeff, score, latent] = pca(X);

D. coeff = pca(X, 'coeff');

Solution

1. Step 1: Recall MATLAB PCA syntax

   The function pca returns coefficients, scores, and variances as outputs.

2. Step 2: Match correct syntax

   Only [coeff, score, latent] = pca(X); correctly calls PCA and stores outputs.

3. Final Answer:

   [coeff, score, latent] = pca(X); -> Option C

4. Quick Check:

   Correct PCA syntax = [coeff, score, latent] = pca(X); [OK]

Hint: Use brackets to capture multiple PCA outputs [OK]

Common Mistakes:

• Using dot notation on matrix
• Passing output names as strings
• Assigning only one output without brackets

3.

Given the MATLAB code below, what is the size of score if X is a 100x5 matrix?

 [coeff, score, latent] = pca(X); 

medium

A. 5x5

B. 100x5

C. 100x100

D. 5x100

Solution

1. Step 1: Understand PCA output sizes

   score contains transformed data points in principal component space, same rows as X.

2. Step 2: Match dimensions

   Since X is 100x5, score is 100x5 (100 samples, 5 components).

3. Final Answer:

   100x5 -> Option B

4. Quick Check:

   score size = samples x components = 100x5 [OK]

Hint: Score matches rows of X and number of components [OK]

Common Mistakes:

• Confusing score size with coeff size
• Assuming score is square matrix
• Swapping rows and columns

4.

Identify the error in this MATLAB PCA code snippet:

 X = rand(50,3);
 [coeff, score, latent] = pca(X);
 disp(score(1,4)); 

medium

A. Indexing score at column 4 causes an error

B. Missing semicolon after rand function

C. Incorrect function name, should be pca_analysis

D. Matrix X size is invalid for PCA

Solution

1. Step 1: Check size of PCA outputs

   Input X has 3 columns, so score has 3 columns (components).

2. Step 2: Identify indexing error

   Accessing score(1,4) tries to get 4th column which does not exist, causing an error.

3. Final Answer:

   Indexing score at column 4 causes an error -> Option A

4. Quick Check:

   score columns = 3, index 4 invalid [OK]

Hint: Check matrix size before indexing columns [OK]

Common Mistakes:

• Assuming more components than input columns
• Ignoring MATLAB indexing errors
• Thinking semicolon omission causes error here

5.

You want to reduce a dataset X (200x10) to 2 principal components using MATLAB PCA. Which code correctly achieves this?

 % Options:
 A) [coeff, score] = pca(X, 'NumComponents', 2);
 B) [coeff, score] = pca(X, 'NumComponents', 2); score = score(:,3:4);
 C) [coeff, score] = pca(X); score = score(:,11:12);
 D) [coeff, score] = pca(X, 'Components', 2); 

hard

A. [coeff, score] = pca(X, 'Components', 2);

B. [coeff, score] = pca(X, 'NumComponents', 2); score = score(:,3:4);

C. [coeff, score] = pca(X); score = score(:,11:12);

D. [coeff, score] = pca(X, 'NumComponents', 2);

Solution

1. Step 1: Understand PCA dimension reduction parameter

   MATLAB's pca accepts 'NumComponents' to limit output components.

2. Step 2: Identify correct usage

   [coeff, score] = pca(X, 'NumComponents', 2); uses correct parameter 'NumComponents', 2 to get 2 components directly.

3. Step 3: Check other options

   B: [coeff, score] = pca(X, 'NumComponents', 2); score = score(:,3:4); tries to slice non-existent columns 3-4; C: [coeff, score] = pca(X); score = score(:,11:12); tries to slice non-existent columns 11-12; D: [coeff, score] = pca(X, 'Components', 2); uses invalid parameter.

4. Final Answer:

   [coeff, score] = pca(X, 'NumComponents', 2); -> Option D

5. Quick Check:

   Use 'NumComponents' to limit PCA output [OK]

Hint: Use 'NumComponents' to get desired PCA components [OK]

Common Mistakes:

• Using wrong parameter name 'Components'
• Using invalid slice indices on PCA score
• Assuming PCA auto-limits components without parameter