Tensor operations (add, mul, matmul) in PyTorch

Introduction

Tensors are like multi-dimensional arrays. We use operations like add, mul, and matmul to combine or transform data in machine learning.

Adding two images pixel by pixel to blend them.

Multiplying features by weights element-wise in a neural network.

Calculating the output of a layer by matrix multiplying inputs and weights.

Combining sensor data from different sources by adding their tensors.

Performing batch operations on data for faster training.

Syntax

PyTorch

import torch

# Add two tensors
result = torch.add(tensor1, tensor2)

# Multiply two tensors element-wise
result = torch.mul(tensor1, tensor2)

# Matrix multiply two tensors
result = torch.matmul(tensor1, tensor2)

All tensors must have compatible shapes for these operations.

Matrix multiplication follows linear algebra rules, not element-wise multiplication.

Examples

Adds two 1D tensors element-wise: [1+4, 2+5, 3+6] = [5, 7, 9]

PyTorch

a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])
add_result = torch.add(a, b)

Multiplies two 1D tensors element-wise: [1*4, 2*5, 3*6] = [4, 10, 18]

PyTorch

a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])
mul_result = torch.mul(a, b)

Matrix multiplies two 2x2 tensors following linear algebra rules.

PyTorch

a = torch.tensor([[1, 2], [3, 4]])
b = torch.tensor([[5, 6], [7, 8]])
matmul_result = torch.matmul(a, b)

Sample Model

This program shows how to add, multiply element-wise, and matrix multiply two 2x2 tensors.

PyTorch

import torch

# Define two tensors
x = torch.tensor([[1, 2], [3, 4]])
y = torch.tensor([[5, 6], [7, 8]])

# Add tensors
add = torch.add(x, y)

# Multiply tensors element-wise
mul = torch.mul(x, y)

# Matrix multiply tensors
matmul = torch.matmul(x, y)

print('Add result:\n', add)
print('Mul result:\n', mul)
print('Matmul result:\n', matmul)

Important Notes

Use torch.add and torch.mul for element-wise operations.

Use torch.matmul for matrix multiplication, which is different from element-wise multiplication.

Shapes must match or be broadcastable for add and mul; for matmul, inner dimensions must align.

Summary

Tensors hold data in multi-dimensional arrays.

Use add and mul for element-wise operations.

Use matmul for matrix multiplication following linear algebra.