QR decomposition in SciPy - Time & Space Complexity

Time Complexity: QR decomposition

O(n^3)

Understanding Time Complexity

We want to understand how the time needed to perform QR decomposition changes as the size of the matrix grows.

Specifically, how does the work increase when we have bigger matrices?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import numpy as np
from scipy.linalg import qr

A = np.random.rand(100, 100)
Q, R = qr(A)
    

This code creates a 100x100 matrix and computes its QR decomposition using SciPy.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Multiplying and subtracting vectors and matrices during the decomposition process.
• How many times: These operations happen repeatedly for each column and row, roughly proportional to the cube of the matrix size.

How Execution Grows With Input

As the matrix size grows, the number of calculations grows much faster.

Input Size (n x n)	Approx. Operations
10 x 10	About 1,000 operations
100 x 100	About 1,000,000 operations
1000 x 1000	About 1,000,000,000 operations

Pattern observation: When the matrix size doubles, the work increases roughly by eight times.

Final Time Complexity

Time Complexity: O(n^3)

This means the time needed grows roughly with the cube of the matrix size, so bigger matrices take much longer to process.

Common Mistake

[X] Wrong: "QR decomposition takes time proportional to the number of elements in the matrix (n squared)."

[OK] Correct: The process involves many matrix multiplications and vector operations that multiply the work, making it grow faster than just the number of elements.

Interview Connect

Understanding how QR decomposition scales helps you explain the cost of matrix operations clearly, a useful skill in data science and engineering roles.

Self-Check

"What if we only decomposed a rectangular matrix with many more rows than columns? How would the time complexity change?"

Practice

1. What does QR decomposition do to a matrix A?

easy

A. Converts A into a lower triangular matrix only

B. Splits A into two diagonal matrices

C. Splits A into an orthogonal matrix Q and an upper triangular matrix R

D. Finds the inverse of A directly

Solution

1. Step 1: Understand QR decomposition definition

   QR decomposition splits a matrix into two parts: an orthogonal matrix Q and an upper triangular matrix R.

2. Step 2: Compare options with definition

   Only Splits A into an orthogonal matrix Q and an upper triangular matrix R correctly describes this split. Others describe different matrix operations.

3. Final Answer:

   Splits A into an orthogonal matrix Q and an upper triangular matrix R -> Option C

4. Quick Check:

   QR decomposition = Q and R matrices [OK]

Hint: QR always produces Q (orthogonal) and R (upper triangular) [OK]

Common Mistakes:

• Confusing QR with LU decomposition
• Thinking QR produces diagonal matrices
• Assuming QR finds matrix inverse

2. Which of the following is the correct way to import and use QR decomposition from scipy?

easy

A. from scipy.linalg import qr
Q, R = qr(A)

B. import scipy.qr
Q, R = scipy.qr(A)

C. from scipy import qr
Q, R = qr(A)

D. import qr from scipy.linalg
Q, R = qr(A)

Solution

1. Step 1: Recall scipy QR import syntax

   The correct import is from scipy.linalg import qr.

2. Step 2: Check usage correctness

   Calling qr(A) returns Q and R. from scipy.linalg import qr Q, R = qr(A) matches this exactly.

3. Final Answer:

   from scipy.linalg import qr Q, R = qr(A) -> Option A

4. Quick Check:

   Correct import and call = from scipy.linalg import qr Q, R = qr(A) [OK]

Hint: Use 'from scipy.linalg import qr' to access QR function [OK]

Common Mistakes:

• Trying to import qr directly from scipy
• Using incorrect import syntax
• Calling qr without importing

3. Given the matrix A = [[1, 2], [3, 4]], what is the shape of Q and R after Q, R = qr(A)?

medium

A. Q is (2, 2) and R is (2, 1)

B. Q is (1, 2) and R is (2, 2)

C. Q is (2, 1) and R is (1, 2)

D. Q is (2, 2) and R is (2, 2)

Solution

1. Step 1: Check input matrix shape

   Matrix A has shape (2, 2).

2. Step 2: Understand QR output shapes

   For a square matrix, Q and R both have the same shape as A, which is (2, 2).

3. Final Answer:

   Q is (2, 2) and R is (2, 2) -> Option D

4. Quick Check:

   Square matrix QR outputs same shape matrices [OK]

Hint: Q and R shapes match input matrix shape for square matrices [OK]

Common Mistakes:

• Assuming R has fewer columns
• Confusing Q shape with R shape
• Thinking Q or R is 1D

4. What is wrong with this code snippet for QR decomposition?

from scipy.linalg import qr
A = [[1, 2], [3, 4]]
Q, R = qr(A)
print(Q)

medium

A. Nothing is wrong with this code snippet

B. The qr function does not return two values

C. Matrix A should be a numpy array, not a list of lists

D. Import statement is incorrect

Solution

1. Step 1: Verify import statement

   The import from scipy.linalg import qr is correct.

2. Step 2: Check qr function returns

   qr(A) returns two values Q and R.

3. Step 3: Validate input type

   List of lists for A works as qr accepts array-like inputs and converts internally.

4. Step 4: Check print statement

   print(Q) correctly outputs the Q matrix.

5. Final Answer:

   Nothing is wrong with this code snippet -> Option A

6. Quick Check:

   Code runs successfully, produces correct Q and R [OK]

Hint: scipy.linalg.qr accepts lists of lists as array-like input [OK]

Common Mistakes:

• Thinking Python lists cannot be passed to qr
• Misunderstanding qr return values
• Incorrect import statements

5. You want to solve the system Ax = b using QR decomposition where A is a 3x3 matrix and b is a vector. Which step correctly describes how to find x after computing Q, R = qr(A)?

hard

A. Compute x = Q @ b then solve Ry = x

B. Compute y = Q.T @ b then solve Rx = y using back substitution

C. Compute y = R @ b then solve Qx = y

D. Compute x = R.T @ Q @ b

Solution

1. Step 1: Use QR decomposition to rewrite Ax = b

   Since A = QR, rewrite as QRx = b.

2. Step 2: Multiply both sides by Q.T

   Multiply by Q.T to get Rx = Q.T b. Define y = Q.T b.

3. Step 3: Solve upper triangular system

   Now solve Rx = y using back substitution to find x.

4. Final Answer:

   Compute y = Q.T @ b then solve Rx = y using back substitution -> Option B

5. Quick Check:

   Use Q.T b and solve Rx = y [OK]

Hint: Multiply b by Q.T, then solve Rx = y [OK]

Common Mistakes:

• Multiplying b by Q instead of Q.T
• Trying to solve Qx = y instead of Rx = y
• Ignoring that R is upper triangular