np.einsum() for efficient computation in NumPy - Time & Space Complexity

Time Complexity: np.einsum() for efficient computation

O(n^3)

Understanding Time Complexity

We want to understand how fast np.einsum() runs when doing calculations on arrays.

Specifically, how does the time it takes grow as the size of the arrays grows?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import numpy as np

A = np.random.rand(1000, 1000)
B = np.random.rand(1000, 1000)

# Compute matrix multiplication using einsum
C = np.einsum('ik,kj->ij', A, B)

This code multiplies two 1000x1000 matrices using np.einsum for efficient computation.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Multiplying and summing elements across rows and columns.
• How many times: For each element in the result matrix, it repeats over the shared dimension (1000 times).

How Execution Grows With Input

As the size of the matrices grows, the number of multiplications and sums grows quickly.

Input Size (n)	Approx. Operations
10	10 x 10 x 10 = 1,000
100	100 x 100 x 100 = 1,000,000
1000	1000 x 1000 x 1000 = 1,000,000,000

Pattern observation: The operations grow by the cube of the input size, so tripling the size makes the work much bigger.

Final Time Complexity

Time Complexity: O(n^3)

This means the time to compute grows roughly with the cube of the matrix size, so bigger matrices take much longer.

Common Mistake

[X] Wrong: "np.einsum() always runs faster than other methods regardless of input size."

[OK] Correct: While np.einsum() is efficient, the time still grows quickly with input size because it does many multiplications and sums. For very large inputs, it still takes a lot of time.

Interview Connect

Understanding how np.einsum() scales helps you explain efficient array operations clearly and confidently in real projects or interviews.

Self-Check

"What if we changed the operation to element-wise multiplication instead of matrix multiplication? How would the time complexity change?"

Practice

1. What is the main purpose of np.einsum() in numpy?

easy

A. To perform efficient and readable array operations using Einstein summation notation

B. To create random arrays with specific distributions

C. To sort arrays in ascending order

D. To reshape arrays into different dimensions

Solution

1. Step 1: Understand the function's goal

   np.einsum() uses Einstein summation notation to express complex array operations clearly and efficiently.

2. Step 2: Compare with other options

   Options B, C, and D describe unrelated numpy functions like random generation, sorting, and reshaping.

3. Final Answer:

   To perform efficient and readable array operations using Einstein summation notation -> Option A

4. Quick Check:

   Purpose of np.einsum() = To perform efficient and readable array operations using Einstein summation notation [OK]

Hint: Remember: einsum is for smart, fast array math [OK]

Common Mistakes:

• Confusing einsum with random array creation
• Thinking einsum sorts or reshapes arrays
• Assuming einsum is only for dot products

2. Which of the following is the correct syntax to compute the matrix product of two 2D arrays A and B using np.einsum()?

easy

A. np.einsum('ij,jk->ik', A, B)

B. np.einsum('ik,jk->ij', A, B)

C. np.einsum('ij,ik->jk', A, B)

D. np.einsum('ji,kj->ik', A, B)

Solution

1. Step 1: Recall matrix multiplication indices

   Matrix multiplication of A (shape ixj) and B (shape jxk) results in shape ixk, summing over j.

2. Step 2: Match einsum subscripts

   np.einsum('ij,jk->ik', A, B) uses 'ij,jk->ik', which correctly sums over j and produces ixk output.

3. Final Answer:

   np.einsum('ij,jk->ik', A, B) -> Option A

4. Quick Check:

   Matrix product einsum = np.einsum('ij,jk->ik', A, B) [OK]

Hint: Use 'ij,jk->ik' for matrix multiplication [OK]

Common Mistakes:

• Mixing up indices order in the subscript string
• Using wrong output indices after ->
• Confusing which axis to sum over

3. Given the code:

import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])
result = np.einsum('ij,ij->i', A, B)
print(result)

What is the output?

medium

A. [19 43]

B. [17 43]

C. [23 53]

D. [17 53]

Solution

1. Step 1: Understand the einsum subscripts 'ij,ij->i'

   This sums element-wise multiplication over the second index j, keeping the first index i.

2. Step 2: Calculate for each i

   For i=0: (1*5)+(2*6)=5+12=17; For i=1: (3*7)+(4*8)=21+32=53.

3. Final Answer:

   [17 53] -> Option D

4. Quick Check:

   Element-wise sum over j = [17 53] [OK]

Hint: Sum over repeated indices, keep others [OK]

Common Mistakes:

• Summing over wrong axis
• Confusing output indices in einsum
• Calculating element-wise product incorrectly

4. Identify the error in this code snippet:

import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])
result = np.einsum('ij,jk->ik', A, B)
print(result)

and choose the correct fix.

medium

A. Ensure B has shape (2, 2) and use 'ij,jk->ik' as is

B. No error; code runs correctly

C. Swap arrays to np.einsum('jk,ij->ik', B, A)

D. Change subscript to 'ij,ik->jk' to match shapes

Solution

1. Step 1: Check array shapes and einsum subscripts

   A and B both have shape (2,2). The subscript 'ij,jk->ik' expects B's first dimension (j) to match A's second dimension (j), which it does.

2. Step 2: Verify if code runs

   Shapes match perfectly for matrix multiplication. No error; the code runs correctly.

3. Final Answer:

   No error; code runs correctly -> Option B

4. Quick Check:

   No error; code runs correctly [OK]

Hint: Check array shapes match einsum indices [OK]

Common Mistakes:

• Assuming code errors without checking shapes
• Changing subscripts incorrectly
• Swapping arrays unnecessarily

5. You have a 3D array data with shape (10, 5, 4) representing 10 samples, 5 features, and 4 time points. Using np.einsum(), how do you compute the sum of products of features and time points for each sample, resulting in a 1D array of length 10?

hard

A. np.einsum('ijk,ijk->k', data, data)

B. np.einsum('ijk,ijk->j', data, data)

C. np.einsum('ijk,ijk->i', data, data)

D. np.einsum('ijk,ijk->ij', data, data)

Solution

1. Step 1: Understand the dimensions and desired output

   Data shape is (samples=10, features=5, time=4). We want to sum over features and time, keeping samples dimension.

2. Step 2: Use einsum to sum over j and k, keep i

   Subscript 'ijk,ijk->i' multiplies element-wise and sums over j and k, resulting in length 10 array.

3. Final Answer:

   np.einsum('ijk,ijk->i', data, data) -> Option C

4. Quick Check:

   Sum over features and time = np.einsum('ijk,ijk->i', data, data) [OK]

Hint: Sum over repeated indices, keep sample index [OK]

Common Mistakes:

• Summing over wrong axes
• Confusing output indices in einsum
• Expecting 2D output instead of 1D