How to Do Math Operations in PyTorch: Simple Guide

In PyTorch, you perform math operations using torch tensor operators like +, -, *, and / or functions like torch.add(), torch.mul(). These operations work element-wise on tensors and support broadcasting for different shapes.

Syntax

PyTorch math operations can be done using simple operators or torch functions:

• tensor1 + tensor2: Adds two tensors element-wise.
• tensor1 - tensor2: Subtracts element-wise.
• tensor1 * tensor2: Multiplies element-wise.
• tensor1 / tensor2: Divides element-wise.
• torch.add(tensor1, tensor2): Adds tensors using function form.
• torch.mul(tensor1, tensor2): Multiplies tensors using function form.

All tensors must be compatible in shape or broadcastable.

python

import torch

a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])

# Using operators
d = a + b  # addition
f = a * b  # multiplication

# Using torch functions
g = torch.add(a, b)
h = torch.mul(a, b)

Example

This example shows how to add, subtract, multiply, and divide two tensors element-wise using both operators and torch functions.

python

import torch

a = torch.tensor([10, 20, 30])
b = torch.tensor([2, 5, 3])

# Addition
add_op = a + b
add_func = torch.add(a, b)

# Subtraction
sub_op = a - b

# Multiplication
mul_op = a * b
mul_func = torch.mul(a, b)

# Division
div_op = a / b

print('Addition (operator):', add_op)
print('Addition (function):', add_func)
print('Subtraction:', sub_op)
print('Multiplication (operator):', mul_op)
print('Multiplication (function):', mul_func)
print('Division:', div_op)

Output

Addition (operator): tensor([12, 25, 33])
Addition (function): tensor([12, 25, 33])
Subtraction: tensor([ 8, 15, 27])
Multiplication (operator): tensor([20, 100,  90])
Multiplication (function): tensor([20, 100,  90])
Division: tensor([5.0000, 4.0000, 10.0000])

Common Pitfalls

Common mistakes when doing math operations in PyTorch include:

• Trying to operate on tensors with incompatible shapes without broadcasting.
• Using Python scalars instead of tensors, which works but can cause confusion.
• Mixing integer division and expecting float results.
• Not using .float() or .double() when needed for decimal operations.

Always check tensor shapes and types before operations.

python

import torch

# Wrong: incompatible shapes
try:
    a = torch.tensor([1, 2, 3])
    b = torch.tensor([[1, 2], [3, 4]])
    c = a + b  # will raise error
except Exception as e:
    print('Error:', e)

# Right: broadcasting works if compatible
c = a + b.T  # transpose b to shape (2,2) -> (2,2) still incompatible

# Correct broadcasting example
b = torch.tensor([[1], [2], [3]])  # shape (3,1)
c = a + b  # shape (3,3) by broadcasting
print('Broadcasted addition result:', c)

Output

Error: The size of tensor a (3) must match the size of tensor b (2) at non-singleton dimension 1
Broadcasted addition result: tensor([[2, 3, 4],
        [3, 4, 5],
        [4, 5, 6]])

Quick Reference

Operation	Operator	Function
Addition	+	torch.add()
Subtraction	-	torch.sub()
Multiplication	*	torch.mul()
Division	/	torch.div()

Key Takeaways

Use simple operators (+, -, *, /) or torch functions (torch.add, torch.mul) for math on tensors.

Ensure tensors have compatible shapes or use broadcasting to avoid errors.

Convert tensors to float type for decimal division to get accurate results.

Check tensor shapes and types before operations to prevent common mistakes.