Broadcasting in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Broadcasting

Which metric matters for Broadcasting and WHY

Broadcasting is a technique to perform operations on tensors of different shapes without copying data. The key metric to check is correctness of output shape and values. This ensures the operation applied element-wise matches expected results. Efficiency (speed and memory) is also important but correctness is the main metric.

Confusion matrix or equivalent visualization

Broadcasting does not have a confusion matrix like classification. Instead, we visualize shapes and how they align:

    Tensor A shape: (3, 1)
    Tensor B shape: (1, 4)
    Result shape: (3, 4)

    Operation: A + B

    Visualization:
    A: [[a1],    B: [[b1, b2, b3, b4]]
        [a2],
        [a3]]

    Result:
    [[a1+b1, a1+b2, a1+b3, a1+b4],
     [a2+b1, a2+b2, a2+b3, a2+b4],
     [a3+b1, a3+b2, a3+b3, a3+b4]]
    

Precision vs Recall tradeoff (or equivalent) with concrete examples

Broadcasting tradeoff is between memory efficiency and code clarity. Broadcasting avoids copying data, saving memory and speeding up computation. But if shapes are not compatible, it causes errors. So, understanding shape rules is key to avoid bugs.

Example: Adding a (3,1) tensor to a (1,4) tensor works well and is memory efficient. But adding (3,1) to (2,4) fails because shapes don't align.

What "good" vs "bad" metric values look like for this use case

Good: The output tensor has the expected shape and values match element-wise operations. No errors occur. Computation is fast and memory use is low.

Bad: Shape mismatch errors, unexpected output shapes, or incorrect values. Excessive memory use due to manual expansion instead of broadcasting.

Metrics pitfalls

• Assuming broadcasting always works without checking shapes causes runtime errors.
• Misunderstanding shape alignment rules leads to silent bugs with wrong results.
• Expanding tensors manually wastes memory and slows down computation.
• Ignoring broadcasting can cause inefficient code that is hard to maintain.

Self-check

Your code adds a tensor of shape (5,1) to a tensor of shape (5,3) without errors. The output shape is (5,3). Is broadcasting working correctly? Why?

Answer: Yes, broadcasting works correctly. The (5,1) tensor is broadcast along the second dimension to match (5,3). This allows element-wise addition without copying data.

Key Result

Broadcasting correctness is measured by matching output shapes and values, ensuring efficient and error-free tensor operations.