Tensor operations (add, mul, matmul) in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of element-wise addition and multiplication

What is the output of the following PyTorch code?

PyTorch

import torch

a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])
add_result = a + b
mul_result = a * b
print(add_result.tolist(), mul_result.tolist())

A [5, 7, 9] [5, 7, 9]

B [4, 7, 9] [4, 10, 18]

C [5, 7, 9] [4, 10, 18]

D [5, 7, 9] [1, 2, 3]

💡 Hint

Remember that + and * between tensors perform element-wise operations.

The + operator adds corresponding elements: 1+4=5, 2+5=7, 3+6=9. The * operator multiplies element-wise: 1*4=4, 2*5=10, 3*6=18.

❓ Model Choice

intermediate

Choosing the correct operation for matrix multiplication

Which PyTorch operation correctly performs matrix multiplication between two 2D tensors?

A torch.matmul(a, b)

B torch.mul(a, b)

C torch.add(a, b)

D a + b

💡 Hint

Matrix multiplication is not element-wise addition or multiplication.

torch.matmul performs matrix multiplication. torch.add and + do element-wise addition. torch.mul does element-wise multiplication.

🔧 Debug

advanced

Identify the error in tensor multiplication

What error will the following code raise?

PyTorch

import torch

a = torch.tensor([1, 2, 3])
b = torch.tensor([[4, 5, 6], [7, 8, 9]])
result = a * b
print(result)

A ValueError: operands could not be broadcast together with shapes (3,) (2,3)

B No error, outputs element-wise multiplication

C TypeError: unsupported operand type(s) for *: 'list' and 'Tensor'

D RuntimeError: The size of tensor a (3) must match the size of tensor b (2) at non-singleton dimension 0

💡 Hint

Check if the shapes of the tensors are compatible for element-wise multiplication.

No error. Tensor a has shape (3,) and tensor b has shape (2,3). The shapes are broadcastable: PyTorch pads a on the left to (1,3), and the leading dimension 1 broadcasts to 2, resulting in element-wise multiplication with output shape (2,3): tensor([[4,10,18],[14,16,27]]).

❓ Metrics

advanced

Calculate output shape after matrix multiplication

Given two tensors a and b with shapes (4, 3) and (3, 5), what is the shape of torch.matmul(a, b)?

A (4, 5)

B (3, 3)

C (5, 4)

D (4, 3)

💡 Hint

Matrix multiplication shape rule: (m, n) x (n, p) = (m, p).

The inner dimensions 3 match. The output shape is (4, 5).

🧠 Conceptual

expert

Difference between torch.mul and torch.matmul

Which statement correctly describes the difference between torch.mul and torch.matmul?

A torch.mul performs matrix multiplication; torch.matmul performs element-wise multiplication.

B Both perform element-wise multiplication but torch.matmul supports broadcasting.

C Both perform matrix multiplication but torch.mul is faster.

D torch.mul performs element-wise multiplication; torch.matmul performs matrix multiplication following linear algebra rules.

💡 Hint

Think about how each function treats tensor dimensions and multiplication rules.

torch.mul multiplies each element of one tensor by the corresponding element of another tensor (element-wise). torch.matmul performs matrix multiplication, combining rows and columns as in linear algebra.

Practice

1. Which PyTorch function is used for element-wise multiplication of two tensors?

easy

A. torch.matmul()

B. torch.dot()

C. torch.add()

D. torch.mul()

Solution

1. Step 1: Understand element-wise multiplication

   Element-wise multiplication multiplies each element of one tensor by the corresponding element of another tensor.

2. Step 2: Identify the correct PyTorch function

   torch.mul() performs element-wise multiplication, while torch.matmul() is for matrix multiplication.

3. Final Answer:

   torch.mul() -> Option D

4. Quick Check:

   Element-wise multiply = torch.mul() [OK]

Hint: Element-wise multiply uses torch.mul() not matmul [OK]

Common Mistakes:

• Confusing matmul with element-wise multiply
• Using torch.add() instead of multiply
• Trying torch.dot() for element-wise multiply

2. Which of the following is the correct syntax to add two tensors a and b in PyTorch?

easy

A. c = torch.add(a, b)

B. c = a + b + 1

C. c = torch.mul(a, b)

D. c = torch.matmul(a, b)

Solution

1. Step 1: Identify the addition operation

   Adding two tensors element-wise can be done with torch.add(a, b) or a + b.

2. Step 2: Check syntax correctness

   torch.add(a, b) is the correct function call. c = a + b + 1 adds 1 extra, so it's incorrect.

3. Final Answer:

   c = torch.add(a, b) -> Option A

4. Quick Check:

   Tensor add syntax = torch.add(a, b) [OK]

Hint: Use torch.add(a, b) or a + b for tensor addition [OK]

Common Mistakes:

• Adding extra numbers accidentally
• Using mul or matmul instead of add
• Incorrect function call syntax

3. What is the output of the following PyTorch code?

import torch
x = torch.tensor([[1, 2], [3, 4]])
y = torch.tensor([[5, 6], [7, 8]])
z = torch.matmul(x, y)
print(z)

medium

A. tensor([[5, 12], [21, 32]])

B. tensor([[6, 8], [10, 12]])

C. tensor([[19, 22], [43, 50]])

D. tensor([[6, 8], [21, 32]])

Solution

1. Step 1: Perform matrix multiplication manually

   Multiply rows of x by columns of y:
   First row: (1*5 + 2*7) = 19, (1*6 + 2*8) = 22
   Second row: (3*5 + 4*7) = 43, (3*6 + 4*8) = 50

2. Step 2: Confirm output tensor

   The result tensor is [[19, 22], [43, 50]].

3. Final Answer:

   tensor([[19, 22], [43, 50]]) -> Option C

4. Quick Check:

   Matrix multiply result = [[19, 22], [43, 50]] [OK]

Hint: Multiply rows by columns for matmul output [OK]

Common Mistakes:

• Adding elements instead of multiplying
• Confusing element-wise multiply with matmul
• Mixing row and column indices

4. Identify the error in this PyTorch code snippet:

import torch
x = torch.tensor([1, 2, 3])
y = torch.tensor([[4, 5, 6], [7, 8, 9]])
z = torch.matmul(x, y)
print(z)

medium

A. Shape mismatch for matrix multiplication

B. x and y must be the same shape for matmul

C. torch.matmul cannot multiply 1D and 2D tensors

D. No error, code runs fine

Solution

1. Step 1: Check tensor shapes

   x shape is (3,), y shape is (2, 3). For matmul, inner dimensions must match: (N, M) x (M, P).

2. Step 2: Identify mismatch

   Here, x is 1D with length 3, y is 2D with shape (2, 3). The inner dimensions don't align for matmul.

3. Final Answer:

   Shape mismatch for matrix multiplication -> Option A

4. Quick Check:

   Matmul needs matching inner dims = shape mismatch [OK]

Hint: Check tensor shapes before matmul to avoid mismatch [OK]

Common Mistakes:

• Assuming 1D and 2D tensors always multiply
• Ignoring shape rules for matmul
• Thinking matmul works like element-wise multiply

5. Given two tensors a = torch.tensor([[1, 2], [3, 4]]) and b = torch.tensor([[2, 0], [1, 2]]), which code correctly computes the element-wise product and then adds the matrix product of a and b?

hard

A. torch.matmul(torch.mul(a, b), torch.add(a, b))

B. torch.add(torch.mul(a, b), torch.matmul(a, b))

C. torch.mul(torch.add(a, b), torch.matmul(a, b))

D. torch.add(torch.matmul(b, a), torch.mul(a, b))

Solution

1. Step 1: Understand the operations order

   First, compute element-wise product: torch.mul(a, b). Then compute matrix product: torch.matmul(a, b).

2. Step 2: Add the two results

   Use torch.add() to sum the element-wise and matrix products.

3. Final Answer:

   torch.add(torch.mul(a, b), torch.matmul(a, b)) -> Option B

4. Quick Check:

   Element-wise mul + matmul = torch.add(torch.mul(), torch.matmul()) [OK]

Hint: Add element-wise mul and matmul results with torch.add() [OK]

Common Mistakes:

• Mixing order of operations
• Using mul on sums instead of add on products
• Confusing arguments order in add or mul