Tensor math operations in TensorFlow

Introduction

Tensors are like multi-dimensional arrays. Doing math with tensors helps computers learn patterns and solve problems.

When you want to add or multiply data in machine learning models.

When you need to calculate predictions from input data.

When you want to combine or transform data in neural networks.

When you want to compute loss or accuracy during training.

When you want to apply mathematical functions like square root or exponent.

Syntax

TensorFlow

import tensorflow as tf

# Basic math operations
result = tf.add(tensor1, tensor2)  # addition
result = tf.subtract(tensor1, tensor2)  # subtraction
result = tf.multiply(tensor1, tensor2)  # element-wise multiplication
result = tf.divide(tensor1, tensor2)  # element-wise division

# Other operations
result = tf.matmul(tensor1, tensor2)  # matrix multiplication
result = tf.sqrt(tensor)  # square root
result = tf.exp(tensor)  # exponentiation

Use tf.add, tf.subtract, tf.multiply, and tf.divide for element-wise math.

Use tf.matmul for matrix multiplication, which is different from element-wise multiplication.

Examples

Adds two 1D tensors element-wise: [1+4, 2+5, 3+6] = [5, 7, 9]

TensorFlow

a = tf.constant([1, 2, 3])
b = tf.constant([4, 5, 6])
sum_ab = tf.add(a, b)

Performs matrix multiplication of two 2x2 tensors.

TensorFlow

a = tf.constant([[1, 2], [3, 4]])
b = tf.constant([[5, 6], [7, 8]])
product = tf.matmul(a, b)

Computes square root of each element: [3.0, 4.0, 5.0]

TensorFlow

a = tf.constant([9.0, 16.0, 25.0])
sqrt_a = tf.sqrt(a)

Sample Model

This program shows basic tensor math: addition, element-wise multiplication, matrix multiplication, and square root.

TensorFlow

import tensorflow as tf

# Define two tensors
x = tf.constant([[1, 2], [3, 4]], dtype=tf.float32)
y = tf.constant([[5, 6], [7, 8]], dtype=tf.float32)

# Element-wise addition
add_result = tf.add(x, y)

# Element-wise multiplication
mul_result = tf.multiply(x, y)

# Matrix multiplication
matmul_result = tf.matmul(x, y)

# Square root of elements in x
sqrt_x = tf.sqrt(x)

# Print results
print("Addition:\n", add_result.numpy())
print("Element-wise Multiplication:\n", mul_result.numpy())
print("Matrix Multiplication:\n", matmul_result.numpy())
print("Square Root of x:\n", sqrt_x.numpy())

Important Notes

Tensors must have compatible shapes for operations like addition or multiplication.

Matrix multiplication requires the inner dimensions to match (e.g., (2,3) x (3,4)).

Use .numpy() to convert tensors to arrays for easy printing.

Summary

Tensors are multi-dimensional arrays used in machine learning.

Tensor math operations include element-wise and matrix multiplication.

TensorFlow provides simple functions like tf.add and tf.matmul to do these operations.

Practice

1. What does the TensorFlow function tf.add(tensor1, tensor2) do?

easy

A. Adds two tensors element-wise

B. Multiplies two tensors element-wise

C. Performs matrix multiplication of two tensors

D. Subtracts the second tensor from the first

Solution

1. Step 1: Understand the function name and purpose

   The function tf.add is designed to add values, so it performs addition.

2. Step 2: Check the operation type

   In TensorFlow, tf.add adds two tensors element-wise, meaning it adds corresponding elements from both tensors.

3. Final Answer:

   Adds two tensors element-wise -> Option A

4. Quick Check:

   tf.add = element-wise addition [OK]

Hint: Add means element-wise sum, not matrix multiply [OK]

Common Mistakes:

• Confusing tf.add with matrix multiplication
• Thinking tf.add subtracts tensors
• Assuming tf.add multiplies tensors

2. Which of the following is the correct syntax to perform matrix multiplication of two tensors a and b in TensorFlow?

easy

A. tf.multiply(a, b)

B. tf.add(a, b)

C. tf.matmul(a, b)

D. a.dot(b)

Solution

1. Step 1: Identify the function for matrix multiplication

   TensorFlow uses tf.matmul specifically for matrix multiplication.

2. Step 2: Check other options

   tf.multiply does element-wise multiplication, tf.add adds tensors, and a.dot(b) is invalid since tf.Tensor has no .dot method.

3. Final Answer:

   tf.matmul(a, b) -> Option C

4. Quick Check:

   Matrix multiply = tf.matmul [OK]

Hint: Matrix multiply uses tf.matmul, not tf.multiply [OK]

Common Mistakes:

• Using tf.multiply for matrix multiplication
• Using a.dot(b) like in NumPy
• Confusing addition with multiplication

3. What is the output of the following TensorFlow code?

import tensorflow as tf
x = tf.constant([[1, 2], [3, 4]])
y = tf.constant([[5, 6], [7, 8]])
result = tf.add(x, y)
print(result.numpy())

medium

A. [[6 12]
[10 32]]

B. [[6 8]
[10 12]]

C. [[5 12]
[21 32]]

D. [[1 2]
[3 4]]

Solution

1. Step 1: Understand the operation

   The code uses tf.add to add two 2x2 tensors element-wise.

2. Step 2: Calculate element-wise addition

   Adding corresponding elements: [[1+5, 2+6], [3+7, 4+8]] = [[6, 8], [10, 12]]

3. Final Answer:

   [[6 8] [10 12]] -> Option B

4. Quick Check:

   Element-wise add = [[6 8] [10 12]] [OK]

Hint: Add each element pair to get the result [OK]

Common Mistakes:

• Confusing element-wise add with matrix multiply
• Adding rows or columns incorrectly
• Printing tensor object instead of numpy array

4. Identify the error in this TensorFlow code snippet and choose the fix:

import tensorflow as tf
x = tf.constant([[1, 2], [3, 4]])
y = tf.constant([5, 6])
result = tf.matmul(x, y)
print(result.numpy())

medium

A. No error, code runs fine

B. Use tf.add instead of tf.matmul

C. Change x to a 1D tensor

D. Change y to a 2x1 tensor: tf.constant([[5], [6]])

Solution

1. Step 1: Understand matrix multiplication shape rules

   x is shape (2,2), y is shape (2,). TensorFlow tf.matmul requires both inputs to be at least rank 2 tensors.

2. Step 2: Identify the error

   Passing a 1D tensor y to tf.matmul causes a shape error because tf.matmul expects rank >= 2 tensors.

3. Step 3: Fix the error

   Change y to a 2D tensor with shape (2,1): tf.constant([[5], [6]]) to make matrix multiplication valid.

4. Final Answer:

   Change y to a 2x1 tensor: tf.constant([[5], [6]]) -> Option D

5. Quick Check:

   tf.matmul requires rank 2 tensors [OK]

Hint: tf.matmul requires 2D tensors; reshape 1D vector to 2D [OK]

Common Mistakes:

• Assuming 1D tensors cause no shape errors in matmul
• Unnecessarily reshaping y to 2D
• Confusing matmul with element-wise operations

5. You have two tensors:
a = tf.constant([[1, 2], [3, 4]])
b = tf.constant([[2, 0], [1, 2]])
Which TensorFlow operation will give the element-wise product of a and b?

hard

A. tf.multiply(a, b)

B. tf.matmul(a, b)

C. tf.add(a, b)

D. tf.tensordot(a, b, axes=1)

Solution

1. Step 1: Understand element-wise product

   Element-wise product multiplies each element of a with the corresponding element of b.

2. Step 2: Identify TensorFlow function for element-wise multiplication

   tf.multiply performs element-wise multiplication, while tf.matmul does matrix multiplication.

3. Final Answer:

   tf.multiply(a, b) -> Option A

4. Quick Check:

   Element-wise multiply = tf.multiply [OK]

Hint: Use tf.multiply for element-wise product, not tf.matmul [OK]

Common Mistakes:

• Using tf.matmul instead of tf.multiply
• Confusing addition with multiplication
• Using tf.tensordot incorrectly