Broadcasting rules in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

TensorFlow Broadcasting Output Shape

Given the following TensorFlow code, what is the shape of the resulting tensor after the addition?

TensorFlow

import tensorflow as tf

x = tf.constant([[1, 2, 3], [4, 5, 6]])  # shape (2, 3)
y = tf.constant([10, 20, 30])             # shape (3,)
result = x + y
result_shape = result.shape

A (2,)

B (3, 2)

C (2, 3)

D (3,)

💡 Hint

Remember how TensorFlow broadcasts smaller tensors to match larger ones by adding dimensions.

The tensor y with shape (3,) is broadcasted to (1, 3) and then to (2, 3) to match x. So the result shape is (2, 3).

🧠 Conceptual

intermediate

Understanding Broadcasting Compatibility

Which pair of tensor shapes can be broadcast together in TensorFlow without error?

A (5, 4) and (5, 4, 1)

B (4, 1, 3) and (3, 1)

C (2, 3) and (3, 2)

D (1, 2, 3) and (3, 1)

💡 Hint

Broadcasting compares shapes from the rightmost dimension to the left.

Shapes (4,1,3) and (3,1) align as (4,1,3) and (1,3,1) after prepending 1. Dimensions are compatible if equal or one is 1.

❓ Hyperparameter

advanced

Broadcasting in Tensor Operations

You have a tensor A of shape (8, 1, 6) and tensor B of shape (1, 5, 6). What will be the shape of A * B after broadcasting in TensorFlow?

A (8, 5, 6)

B (8, 5)

C (1, 5, 6)

D (8, 1, 6)

💡 Hint

Broadcasting expands dimensions where size is 1 to match the other tensor.

Tensor A shape (8,1,6) and B shape (1,5,6) broadcast to (8,5,6) by expanding the 1s.

🔧 Debug

advanced

Identifying Broadcasting Error

What error will this TensorFlow code raise due to broadcasting rules?

TensorFlow

import tensorflow as tf

x = tf.constant([[1, 2], [3, 4]])  # shape (2, 2)
y = tf.constant([1, 2, 3])          # shape (3,)
result = x + y

A No error, result shape is (2, 3)

B ValueError: operands could not be broadcast together with shapes (2,2) (3,)

C TypeError: unsupported operand type(s) for +: 'Tensor' and 'Tensor'

D InvalidArgumentError: Incompatible shapes: [2,2] vs. [3]

💡 Hint

Check if the dimensions can align from right to left for broadcasting.

Shapes (2,2) and (3,) cannot be broadcast because the last dimensions 2 and 3 differ and neither is 1.

❓ Model Choice

expert

Choosing Tensor Shapes for Efficient Broadcasting

You want to add two tensors efficiently in TensorFlow with minimal memory overhead. Which pair of shapes is best for broadcasting?

A (64, 1, 1) and (1, 32, 128)

B (64, 32, 128) and (64, 32, 128)

C (64, 32, 128) and (32, 128)

D (64, 1, 128) and (1, 32, 128)

💡 Hint

Smaller dimensions with 1 allow broadcasting without copying large data.

Option A uses many 1s allowing broadcasting with minimal memory use. Option A duplicates full shape, no broadcasting benefit.

Practice

1. What does broadcasting in TensorFlow allow you to do?

easy

A. Perform math operations on tensors with different but compatible shapes

B. Convert tensors into Python lists automatically

C. Change the data type of tensors without copying data

D. Create new tensors with random values

Solution

1. Step 1: Understand broadcasting concept

   Broadcasting lets TensorFlow perform element-wise operations on tensors even if their shapes differ, as long as they are compatible.

2. Step 2: Identify the correct description

   Only Perform math operations on tensors with different but compatible shapes correctly describes this feature; others describe unrelated tensor operations.

3. Final Answer:

   Perform math operations on tensors with different but compatible shapes -> Option A

4. Quick Check:

   Broadcasting = math on compatible shapes [OK]

Hint: Broadcasting = math on tensors with compatible shapes [OK]

Common Mistakes:

• Thinking broadcasting changes data types
• Confusing broadcasting with tensor creation
• Assuming broadcasting converts tensors to lists

2. Which of the following TensorFlow code snippets correctly broadcasts a tensor of shape (3, 1) with a tensor of shape (1, 4)?

easy

A. tf.constant([1, 2, 3]) + tf.constant([4, 5, 6, 7])

B. tf.constant([[1], [2], [3]]) + tf.constant([[4, 5, 6, 7]])

C. tf.constant([[1, 2, 3]]) + tf.constant([[4], [5], [6], [7]])

D. tf.constant([[1], [2], [3]]) + tf.constant([[4], [5], [6], [7]])

Solution

1. Step 1: Check shapes of tensors in each option

   tf.constant([[1], [2], [3]]) + tf.constant([[4, 5, 6, 7]]) adds (3,1) tensor to (1,4) tensor, which are compatible for broadcasting.

2. Step 2: Verify broadcasting rules

   Shapes (3,1) and (1,4) broadcast to (3,4). Other options have incompatible shapes or wrong dimensions.

3. Final Answer:

   tf.constant([[1], [2], [3]]) + tf.constant([[4, 5, 6, 7]]) -> Option B

4. Quick Check:

   Shapes (3,1) + (1,4) broadcast correctly [OK]

Hint: Match trailing dims: 1 and N broadcast fine [OK]

Common Mistakes:

• Ignoring shape dimensions order
• Assuming 1D tensors broadcast like 2D
• Mixing up rows and columns in shapes

3. What is the output shape of the following TensorFlow operation?

import tensorflow as tf
x = tf.constant([[1, 2, 3]])  # shape (1, 3)
y = tf.constant([4, 5, 6, 7])  # shape (4,)
z = x + y

medium

A. (1, 3, 4)

B. (4, 3)

C. (1, 4)

D. Error due to incompatible shapes

Solution

1. Step 1: Analyze shapes of x and y

   x has shape (1,3), y has shape (4,). TensorFlow aligns shapes from the right.

2. Step 2: Apply broadcasting rules

   y's shape (4,) is treated as (1,4). Shapes (1,3) and (1,4) are incompatible because 3 != 4 and neither is 1.

3. Step 3: Check if broadcasting possible

   Since last dimensions differ and neither is 1, broadcasting fails, causing an error.

4. Final Answer:

   Error due to incompatible shapes -> Option D

5. Quick Check:

   Incompatible shapes cause error [OK]

Hint: Broadcast dims must be equal or 1 from right [OK]

Common Mistakes:

• Assuming (4,) broadcasts to (3,)
• Ignoring dimension order in broadcasting
• Expecting automatic reshaping without error

4. You have two tensors:
a = tf.constant([[1, 2, 3], [4, 5, 6]]) (shape (2, 3))
b = tf.constant([1, 2]) (shape (2,))
Why does a + b raise an error, and how can you fix it?

medium

A. Shapes are incompatible; reshape b to (2,1) before adding

B. Data types differ; cast b to a's dtype

C. Tensors must be same shape; reshape a to (2,2)

D. Broadcasting always works; error is from another cause

Solution

1. Step 1: Check shapes of a and b

   a is (2,3), b is (2,). Broadcasting compares from right: 3 vs 2 incompatible.

2. Step 2: Fix shape for broadcasting

   Reshape b to (2,1) so shapes become (2,3) and (2,1), which broadcast to (2,3).

3. Final Answer:

   Shapes are incompatible; reshape b to (2,1) before adding -> Option A

4. Quick Check:

   Reshape b to (2,1) fixes broadcasting [OK]

Hint: Match dims from right; add missing dims with reshape [OK]

Common Mistakes:

• Ignoring shape mismatch causes error
• Trying to reshape a incorrectly
• Assuming broadcasting fixes all shape issues automatically

5. Given a tensor t of shape (5, 1, 3), you want to add a bias tensor b of shape (3,) to each element along the last dimension. Which code correctly applies broadcasting to add b to t?

hard

A. t + tf.reshape(b, (3, 1))

B. t + tf.reshape(b, (3, 1, 1))

C. t + tf.reshape(b, (1, 1, 3))

D. t + tf.reshape(b, (5, 1, 3))

Solution

1. Step 1: Understand shapes and broadcasting

   t shape is (5,1,3), b shape is (3,). To add b along last dim, b must broadcast to (5,1,3).

2. Step 2: Reshape b for broadcasting

   Reshape b to (1,1,3) so it broadcasts correctly across first two dims.

3. Step 3: Check other options

   A reshapes to (3,1), which pads to (1,3,1) and mismatches middle dim. B reshapes to (3,1,1), mismatching first dim. D fails due to element count mismatch (3 vs 15).

4. Final Answer:

   t + tf.reshape(b, (1, 1, 3)) -> Option C

5. Quick Check:

   Reshape bias to (1,1,3) for last-dim addition [OK]

Hint: Reshape bias to add dims before last dimension [OK]

Common Mistakes:

• Not reshaping bias tensor correctly
• Assuming 1D tensor broadcasts without reshape
• Reshaping bias to wrong shape