Tensor math operations in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Tensor math operations

Which metric matters for Tensor math operations and WHY

Tensor math operations are the building blocks of machine learning models. The key metrics to check here are computational correctness and performance efficiency. Correctness means the math results (like sums, products) must be accurate. Efficiency means operations should run fast and use memory well. These metrics matter because wrong math breaks the model, and slow math makes training too long.

Confusion matrix or equivalent visualization

For tensor math, we don't use confusion matrices. Instead, we verify results by comparing expected and actual tensor outputs. For example:

Input A: [1, 2, 3]
Input B: [4, 5, 6]
Operation: Element-wise addition
Expected output: [5, 7, 9]
Actual output: [5, 7, 9]

If expected equals actual (within floating point tolerance), the operation is correct.

Tradeoff: Precision vs Performance

Tensor math operations often trade off between precision (how exact the numbers are) and performance (speed and memory use). For example, using 32-bit floats is faster but less precise than 64-bit floats. In some cases, lower precision is fine and speeds up training. In others, like scientific data, high precision is needed to avoid errors.

What "good" vs "bad" metric values look like

Good: Tensor operations produce exact expected results (within floating point tolerance), run quickly, and use reasonable memory.

Bad: Results differ from expected (wrong sums, products), operations are slow, or use too much memory causing crashes.

Common pitfalls in tensor math metrics

• Ignoring floating point rounding errors and expecting exact equality.
• Using wrong tensor shapes causing silent broadcasting errors.
• Overlooking performance bottlenecks by not profiling operations.
• Mixing data types (int vs float) leading to unexpected results.

Self-check question

Your tensor addition operation returns [5.0001, 7.0002, 9.0001] instead of [5, 7, 9]. Is this good? Why?

Answer: Yes, this is good because small differences like 0.0001 are normal due to floating point precision limits. The operation is effectively correct.

Key Result

Tensor math operations must be accurate within floating point tolerance and efficient in speed and memory.

Practice

1. What does the TensorFlow function tf.add(tensor1, tensor2) do?

easy

A. Adds two tensors element-wise

B. Multiplies two tensors element-wise

C. Performs matrix multiplication of two tensors

D. Subtracts the second tensor from the first

Solution

1. Step 1: Understand the function name and purpose

   The function tf.add is designed to add values, so it performs addition.

2. Step 2: Check the operation type

   In TensorFlow, tf.add adds two tensors element-wise, meaning it adds corresponding elements from both tensors.

3. Final Answer:

   Adds two tensors element-wise -> Option A

4. Quick Check:

   tf.add = element-wise addition [OK]

Hint: Add means element-wise sum, not matrix multiply [OK]

Common Mistakes:

• Confusing tf.add with matrix multiplication
• Thinking tf.add subtracts tensors
• Assuming tf.add multiplies tensors

2. Which of the following is the correct syntax to perform matrix multiplication of two tensors a and b in TensorFlow?

easy

A. tf.multiply(a, b)

B. tf.add(a, b)

C. tf.matmul(a, b)

D. a.dot(b)

Solution

1. Step 1: Identify the function for matrix multiplication

   TensorFlow uses tf.matmul specifically for matrix multiplication.

2. Step 2: Check other options

   tf.multiply does element-wise multiplication, tf.add adds tensors, and a.dot(b) is invalid since tf.Tensor has no .dot method.

3. Final Answer:

   tf.matmul(a, b) -> Option C

4. Quick Check:

   Matrix multiply = tf.matmul [OK]

Hint: Matrix multiply uses tf.matmul, not tf.multiply [OK]

Common Mistakes:

• Using tf.multiply for matrix multiplication
• Using a.dot(b) like in NumPy
• Confusing addition with multiplication

3. What is the output of the following TensorFlow code?

import tensorflow as tf
x = tf.constant([[1, 2], [3, 4]])
y = tf.constant([[5, 6], [7, 8]])
result = tf.add(x, y)
print(result.numpy())

medium

A. [[6 12]
[10 32]]

B. [[6 8]
[10 12]]

C. [[5 12]
[21 32]]

D. [[1 2]
[3 4]]

Solution

1. Step 1: Understand the operation

   The code uses tf.add to add two 2x2 tensors element-wise.

2. Step 2: Calculate element-wise addition

   Adding corresponding elements: [[1+5, 2+6], [3+7, 4+8]] = [[6, 8], [10, 12]]

3. Final Answer:

   [[6 8] [10 12]] -> Option B

4. Quick Check:

   Element-wise add = [[6 8] [10 12]] [OK]

Hint: Add each element pair to get the result [OK]

Common Mistakes:

• Confusing element-wise add with matrix multiply
• Adding rows or columns incorrectly
• Printing tensor object instead of numpy array

4. Identify the error in this TensorFlow code snippet and choose the fix:

import tensorflow as tf
x = tf.constant([[1, 2], [3, 4]])
y = tf.constant([5, 6])
result = tf.matmul(x, y)
print(result.numpy())

medium

A. No error, code runs fine

B. Use tf.add instead of tf.matmul

C. Change x to a 1D tensor

D. Change y to a 2x1 tensor: tf.constant([[5], [6]])

Solution

1. Step 1: Understand matrix multiplication shape rules

   x is shape (2,2), y is shape (2,). TensorFlow tf.matmul requires both inputs to be at least rank 2 tensors.

2. Step 2: Identify the error

   Passing a 1D tensor y to tf.matmul causes a shape error because tf.matmul expects rank >= 2 tensors.

3. Step 3: Fix the error

   Change y to a 2D tensor with shape (2,1): tf.constant([[5], [6]]) to make matrix multiplication valid.

4. Final Answer:

   Change y to a 2x1 tensor: tf.constant([[5], [6]]) -> Option D

5. Quick Check:

   tf.matmul requires rank 2 tensors [OK]

Hint: tf.matmul requires 2D tensors; reshape 1D vector to 2D [OK]

Common Mistakes:

• Assuming 1D tensors cause no shape errors in matmul
• Unnecessarily reshaping y to 2D
• Confusing matmul with element-wise operations

5. You have two tensors:
a = tf.constant([[1, 2], [3, 4]])
b = tf.constant([[2, 0], [1, 2]])
Which TensorFlow operation will give the element-wise product of a and b?

hard

A. tf.multiply(a, b)

B. tf.matmul(a, b)

C. tf.add(a, b)

D. tf.tensordot(a, b, axes=1)

Solution

1. Step 1: Understand element-wise product

   Element-wise product multiplies each element of a with the corresponding element of b.

2. Step 2: Identify TensorFlow function for element-wise multiplication

   tf.multiply performs element-wise multiplication, while tf.matmul does matrix multiplication.

3. Final Answer:

   tf.multiply(a, b) -> Option A

4. Quick Check:

   Element-wise multiply = tf.multiply [OK]

Hint: Use tf.multiply for element-wise product, not tf.matmul [OK]

Common Mistakes:

• Using tf.matmul instead of tf.multiply
• Confusing addition with multiplication
• Using tf.tensordot incorrectly