Tensor math operations in TensorFlow - Model Pipeline Trace

Model Pipeline - Tensor math operations

This pipeline shows how tensors (multi-dimensional arrays) go through math operations like addition, multiplication, and activation functions. These operations prepare data for machine learning models.

Data Flow - 4 Stages

1Input tensor

3 rows x 3 columns→Start with a 3x3 tensor of numbers→3 rows x 3 columns

[[1, 2, 3], [4, 5, 6], [7, 8, 9]]

↓

2Element-wise addition

3 rows x 3 columns→Add a constant tensor [[1,1,1],[1,1,1],[1,1,1]] to input tensor→3 rows x 3 columns

[[2, 3, 4], [5, 6, 7], [8, 9, 10]]

↓

3Element-wise multiplication

3 rows x 3 columns→Multiply tensor by 2 element-wise→3 rows x 3 columns

[[4, 6, 8], [10, 12, 14], [16, 18, 20]]

↓

4Activation function (ReLU)

3 rows x 3 columns→Apply ReLU to set negative values to zero→3 rows x 3 columns

[[4, 6, 8], [10, 12, 14], [16, 18, 20]]

Training Trace - Epoch by Epoch

Loss
0.8 |****
0.6 |***
0.4 |**
0.3 |*
0.25|*
    +------------
    Epochs 1-5

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.8	0.45	Loss starts high, accuracy low as model begins learning
2	0.6	0.60	Loss decreases, accuracy improves as tensor operations help model
3	0.4	0.75	Model learns better features, loss drops further
4	0.3	0.85	Good convergence, tensor math supports learning
5	0.25	0.90	Training stabilizes with low loss and high accuracy

Prediction Trace - 4 Layers

Layer 1: Input tensor

Layer 2: Add constant tensor

Layer 3: Multiply by 2

Layer 4: ReLU activation

Model Quiz - 3 Questions

What happens to each element when we add a constant tensor?

A Elements are replaced by the constant

B Each element is multiplied by the constant

C Each element increases by the constant value

D Elements become zero

Key Insight

Tensor math operations like addition, multiplication, and activation functions transform data step-by-step. These transformations help models learn better by adjusting values and introducing non-linearity.

Practice

1. What does the TensorFlow function tf.add(tensor1, tensor2) do?

easy

A. Adds two tensors element-wise

B. Multiplies two tensors element-wise

C. Performs matrix multiplication of two tensors

D. Subtracts the second tensor from the first

Solution

1. Step 1: Understand the function name and purpose

   The function tf.add is designed to add values, so it performs addition.

2. Step 2: Check the operation type

   In TensorFlow, tf.add adds two tensors element-wise, meaning it adds corresponding elements from both tensors.

3. Final Answer:

   Adds two tensors element-wise -> Option A

4. Quick Check:

   tf.add = element-wise addition [OK]

Hint: Add means element-wise sum, not matrix multiply [OK]

Common Mistakes:

• Confusing tf.add with matrix multiplication
• Thinking tf.add subtracts tensors
• Assuming tf.add multiplies tensors

2. Which of the following is the correct syntax to perform matrix multiplication of two tensors a and b in TensorFlow?

easy

A. tf.multiply(a, b)

B. tf.add(a, b)

C. tf.matmul(a, b)

D. a.dot(b)

Solution

1. Step 1: Identify the function for matrix multiplication

   TensorFlow uses tf.matmul specifically for matrix multiplication.

2. Step 2: Check other options

   tf.multiply does element-wise multiplication, tf.add adds tensors, and a.dot(b) is invalid since tf.Tensor has no .dot method.

3. Final Answer:

   tf.matmul(a, b) -> Option C

4. Quick Check:

   Matrix multiply = tf.matmul [OK]

Hint: Matrix multiply uses tf.matmul, not tf.multiply [OK]

Common Mistakes:

• Using tf.multiply for matrix multiplication
• Using a.dot(b) like in NumPy
• Confusing addition with multiplication

3. What is the output of the following TensorFlow code?

import tensorflow as tf
x = tf.constant([[1, 2], [3, 4]])
y = tf.constant([[5, 6], [7, 8]])
result = tf.add(x, y)
print(result.numpy())

medium

A. [[6 12]
[10 32]]

B. [[6 8]
[10 12]]

C. [[5 12]
[21 32]]

D. [[1 2]
[3 4]]

Solution

1. Step 1: Understand the operation

   The code uses tf.add to add two 2x2 tensors element-wise.

2. Step 2: Calculate element-wise addition

   Adding corresponding elements: [[1+5, 2+6], [3+7, 4+8]] = [[6, 8], [10, 12]]

3. Final Answer:

   [[6 8] [10 12]] -> Option B

4. Quick Check:

   Element-wise add = [[6 8] [10 12]] [OK]

Hint: Add each element pair to get the result [OK]

Common Mistakes:

• Confusing element-wise add with matrix multiply
• Adding rows or columns incorrectly
• Printing tensor object instead of numpy array

4. Identify the error in this TensorFlow code snippet and choose the fix:

import tensorflow as tf
x = tf.constant([[1, 2], [3, 4]])
y = tf.constant([5, 6])
result = tf.matmul(x, y)
print(result.numpy())

medium

A. No error, code runs fine

B. Use tf.add instead of tf.matmul

C. Change x to a 1D tensor

D. Change y to a 2x1 tensor: tf.constant([[5], [6]])

Solution

1. Step 1: Understand matrix multiplication shape rules

   x is shape (2,2), y is shape (2,). TensorFlow tf.matmul requires both inputs to be at least rank 2 tensors.

2. Step 2: Identify the error

   Passing a 1D tensor y to tf.matmul causes a shape error because tf.matmul expects rank >= 2 tensors.

3. Step 3: Fix the error

   Change y to a 2D tensor with shape (2,1): tf.constant([[5], [6]]) to make matrix multiplication valid.

4. Final Answer:

   Change y to a 2x1 tensor: tf.constant([[5], [6]]) -> Option D

5. Quick Check:

   tf.matmul requires rank 2 tensors [OK]

Hint: tf.matmul requires 2D tensors; reshape 1D vector to 2D [OK]

Common Mistakes:

• Assuming 1D tensors cause no shape errors in matmul
• Unnecessarily reshaping y to 2D
• Confusing matmul with element-wise operations

5. You have two tensors:
a = tf.constant([[1, 2], [3, 4]])
b = tf.constant([[2, 0], [1, 2]])
Which TensorFlow operation will give the element-wise product of a and b?

hard

A. tf.multiply(a, b)

B. tf.matmul(a, b)

C. tf.add(a, b)

D. tf.tensordot(a, b, axes=1)

Solution

1. Step 1: Understand element-wise product

   Element-wise product multiplies each element of a with the corresponding element of b.

2. Step 2: Identify TensorFlow function for element-wise multiplication

   tf.multiply performs element-wise multiplication, while tf.matmul does matrix multiplication.

3. Final Answer:

   tf.multiply(a, b) -> Option A

4. Quick Check:

   Element-wise multiply = tf.multiply [OK]

Hint: Use tf.multiply for element-wise product, not tf.matmul [OK]

Common Mistakes:

• Using tf.matmul instead of tf.multiply
• Confusing addition with multiplication
• Using tf.tensordot incorrectly