Matrix multiplication (*) in MATLAB - Deep Dive

Overview - Matrix multiplication (*)

What is it?

Matrix multiplication is a way to combine two matrices to produce a new matrix. It involves multiplying rows of the first matrix by columns of the second matrix and adding the results. This operation is different from multiplying numbers element by element. It is used to transform data, solve systems of equations, and model many real-world problems.

Why it matters

Without matrix multiplication, many calculations in science, engineering, and data analysis would be impossible or very slow. It allows us to represent and compute complex transformations efficiently. For example, in graphics, it helps rotate and scale images; in machine learning, it helps combine data and weights to make predictions.

Where it fits

Before learning matrix multiplication, you should understand what matrices and vectors are, and how to perform basic arithmetic with numbers. After mastering matrix multiplication, you can learn about matrix properties, inverse matrices, and applications like linear regression and neural networks.

Mental Model

Core Idea

Matrix multiplication combines rows of one matrix with columns of another by multiplying and summing to create a new matrix representing combined transformations or relations.

Think of it like...

Imagine you have a recipe book (first matrix) and a list of ingredients (second matrix). To find out how much of each ingredient you need for all recipes, you multiply the amount each recipe requires by the quantity of ingredients available and add them up. The result tells you the total ingredients needed for all recipes combined.

Matrix A (m×n) × Matrix B (n×p) = Matrix C (m×p)

┌─────────┐   ┌─────────┐   ┌─────────┐
│ a11 a12 │   │ b11 b12 │   │ c11 c12 │
│ a21 a22 │ × │ b21 b22 │ = │ c21 c22 │
└─────────┘   └─────────┘   └─────────┘

Each c_ij = sum of (row i of A * column j of B)

Build-Up - 7 Steps

1

Foundation

Understanding matrices and dimensions

Concept: Learn what matrices are and how their size (dimensions) is described.

A matrix is a grid of numbers arranged in rows and columns. The size is written as rows × columns. For example, a 2×3 matrix has 2 rows and 3 columns. You can only multiply two matrices if the number of columns in the first equals the number of rows in the second.

Result

You can identify if two matrices can be multiplied by checking their dimensions.

Knowing matrix dimensions is essential because multiplication rules depend on matching sizes, preventing errors.

2

Foundation

Basic element-wise multiplication vs matrix multiplication

Concept: Distinguish between multiplying matrices element by element and the proper matrix multiplication.

Element-wise multiplication multiplies each element in one matrix by the corresponding element in another matrix of the same size. Matrix multiplication combines rows and columns with sums of products, producing a different sized matrix. In MATLAB, element-wise multiplication uses '.*', while matrix multiplication uses '*'.

Result

You understand the difference and know which operator to use for each.

Confusing these two leads to wrong results; knowing the difference helps apply the right operation.

3

Intermediate

Calculating one element in the product matrix

🤔Before reading on: do you think each element in the product matrix is a simple multiplication or a sum of products? Commit to your answer.

Concept: Each element in the result matrix is the sum of products of corresponding elements from a row of the first matrix and a column of the second.

To find element c_ij in the product matrix, multiply each element in row i of the first matrix by the corresponding element in column j of the second matrix, then add all these products together. For example, c_11 = a_11*b_11 + a_12*b_21 + ...

Result

You can compute any element of the product matrix by following this rule.

Understanding this calculation demystifies matrix multiplication and helps debug or implement it manually.

4

Intermediate

Using MATLAB's * operator for matrix multiplication

Concept: Learn how to perform matrix multiplication in MATLAB using the * operator and what errors to expect.

In MATLAB, use A * B to multiply matrices A and B if their dimensions match (columns of A = rows of B). If they don't match, MATLAB gives an error. For example, if A is 2×3 and B is 3×2, A * B works and results in a 2×2 matrix.

Result

You can multiply matrices in MATLAB and interpret errors related to dimension mismatch.

Knowing MATLAB's behavior prevents confusion and helps write correct matrix operations.

5

Intermediate

Matrix multiplication properties and implications

🤔Before reading on: do you think matrix multiplication is commutative (A*B = B*A)? Commit to your answer.

Concept: Matrix multiplication is associative and distributive but generally not commutative.

You can group multiplications like (A*B)*C = A*(B*C), and distribute over addition like A*(B+C) = A*B + A*C. However, switching order usually changes the result or makes it undefined. For example, A*B may be defined but B*A may not be, or they may produce different matrices.

Result

You understand how matrix multiplication behaves and why order matters.

Recognizing these properties helps avoid mistakes and understand matrix algebra deeply.

6

Advanced

Matrix multiplication in solving linear systems

🤔Before reading on: do you think matrix multiplication can help solve equations with many variables at once? Commit to your answer.

Concept: Matrix multiplication allows expressing and solving multiple linear equations simultaneously using matrix notation.

A system of linear equations can be written as A*x = b, where A is a matrix of coefficients, x is a vector of variables, and b is a vector of constants. Multiplying A by x combines all equations into one matrix equation. Solving for x involves matrix operations like inversion or factorization.

Result

You see how matrix multiplication simplifies complex equation solving.

Understanding this application shows matrix multiplication's power beyond simple arithmetic.

7

Expert

Performance and optimization of matrix multiplication

🤔Before reading on: do you think the order of multiplying multiple matrices affects computation speed? Commit to your answer.

Concept: The order and method of multiplying matrices affect computational efficiency, especially for large matrices.

Matrix multiplication requires many operations. When multiplying several matrices, choosing the order (parentheses placement) can reduce calculations drastically. MATLAB and other tools use optimized algorithms and hardware acceleration to speed up multiplication. Understanding this helps in writing efficient code and handling big data.

Result

You appreciate the importance of optimization in matrix multiplication.

Knowing performance aspects prevents slow programs and guides better algorithm design.

Under the Hood

Matrix multiplication works by taking each row vector from the first matrix and each column vector from the second matrix, multiplying their corresponding elements, and summing these products to produce one element in the result matrix. Internally, MATLAB uses optimized linear algebra libraries that perform these operations efficiently using low-level CPU instructions and memory management.

Why designed this way?

Matrix multiplication is defined this way because it corresponds to composing linear transformations, which is fundamental in mathematics and physics. This definition preserves important properties like associativity and aligns with vector space theory. Alternatives like element-wise multiplication exist but serve different purposes.

┌─────────────┐       ┌─────────────┐       ┌─────────────┐
│ Row i of A  │       │ Column j of B│       │ Element c_ij│
│ a_i1 a_i2 ..│  ×    │ b_1j b_2j ..│  =    │ sum(a_ik*b_kj)│
└─────────────┘       └─────────────┘       └─────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Is matrix multiplication commutative (A*B = B*A)? Commit to yes or no.

Common Belief: Matrix multiplication is just like regular multiplication of numbers, so order does not matter.

Reality: Matrix multiplication is generally not commutative; changing the order can change the result or make the operation invalid.

Why it matters: Assuming commutativity can cause wrong calculations and bugs in programs, especially in transformations and data processing.

Quick: Does the * operator in MATLAB always multiply matrices element-wise? Commit to yes or no.

Common Belief: The * operator multiplies matrices element by element.

Reality: In MATLAB, * performs matrix multiplication, not element-wise multiplication. Element-wise multiplication uses the .* operator.

Why it matters: Using * when element-wise multiplication is needed leads to errors or unexpected results.

Quick: Can you multiply any two matrices regardless of their sizes? Commit to yes or no.

Common Belief: Any two matrices can be multiplied as long as they have the same number of elements.

Reality: Matrix multiplication requires the number of columns in the first matrix to equal the number of rows in the second matrix.

Why it matters: Ignoring dimension rules causes errors and confusion when performing matrix operations.

Quick: Is the element c_ij in the product matrix just the product of a_ij and b_ij? Commit to yes or no.

Common Belief: Each element in the product matrix is the product of corresponding elements from the two matrices.

Reality: Each element in the product matrix is the sum of products of elements from the row of the first matrix and the column of the second matrix, not just a single product.

Why it matters: Misunderstanding this leads to incorrect manual calculations and misuse of matrix operations.

Expert Zone

1

Matrix multiplication order affects not only the result but also computational cost; choosing the right order can optimize performance.

2

Sparse matrices use specialized multiplication algorithms to save memory and speed up calculations, which differ from dense matrix multiplication.

3

Floating-point precision and rounding errors can accumulate in large matrix multiplications, affecting numerical stability in sensitive applications.

When NOT to use

Matrix multiplication is not suitable when you want to multiply matrices element-wise; use element-wise operators instead. Also, for very large sparse matrices, specialized sparse matrix multiplication methods or libraries should be used to improve efficiency.

Production Patterns

In production, matrix multiplication is used in machine learning for forward and backward passes in neural networks, in graphics for transformations, and in simulations for solving systems of equations. Efficient use involves batching operations, leveraging GPU acceleration, and using optimized libraries like BLAS.

Connections

Linear transformations

Matrix multiplication represents the composition of linear transformations.

Understanding matrix multiplication as combining transformations helps grasp its geometric and algebraic significance.

Dot product

Each element in matrix multiplication is a dot product of a row and a column vector.

Knowing the dot product clarifies how matrix multiplication aggregates information from rows and columns.

Computer graphics

Matrix multiplication is used to apply multiple transformations like rotation and scaling to images and 3D models.

Recognizing this connection shows how abstract math powers visual effects and animations.

Common Pitfalls

#1 Trying to multiply matrices with incompatible dimensions.

Wrong approach:

A = rand(2,3);
B = rand(4,2);
C = A * B;

Correct approach:

A = rand(2,3);
B = rand(3,4);
C = A * B;

Root cause:

Not checking that the number of columns in the first matrix matches the number of rows in the second.

#2 Using * operator when element-wise multiplication is intended.

Wrong approach:

A = [1 2; 3 4];
B = [5 6; 7 8];
C = A * B;

Correct approach:

A = [1 2; 3 4];
B = [5 6; 7 8];
C = A .* B;

Root cause:

Confusing matrix multiplication with element-wise multiplication and not using the correct operator.

#3 Assuming matrix multiplication is commutative and swapping order.

Wrong approach:

C1 = A * B;
C2 = B * A; % assuming C1 == C2

Correct approach:

C1 = A * B;
% Do not assume B * A equals C1; check dimensions and results separately.

Root cause:

Misunderstanding that matrix multiplication order affects results and validity.

Key Takeaways

Matrix multiplication combines rows of one matrix with columns of another by multiplying and summing elements to produce a new matrix.

The operation requires the first matrix's columns to match the second matrix's rows, or it will fail.

Matrix multiplication is not commutative; changing the order usually changes the result or makes it invalid.

In MATLAB, use * for matrix multiplication and .* for element-wise multiplication to avoid confusion.

Understanding matrix multiplication is essential for solving systems of equations, performing transformations, and many data science applications.