Broadcasting with higher dimensions in NumPy - Step-by-Step Execution

Concept Flow - Broadcasting with higher dimensions

Start with arrays of different shapes

↓

Compare shapes from trailing dimensions

↓

Check dimension compatibility

↓

Broadcast smaller array

↓

Perform element-wise operation

↓

Result array

Broadcasting compares array shapes from the end, expands smaller arrays by adding dimensions of size 1, then performs element-wise operations.

Execution Sample

NumPy

import numpy as np
A = np.array([[1, 2, 3], [4, 5, 6]])
B = np.array([10, 20, 30])
C = A + B[:, np.newaxis]

Add a 1D array B to a 2D array A by reshaping B to match A's dimensions using broadcasting.

Execution Table

Step	A shape	B shape	B reshaped shape	Broadcasting check	Operation	Result shape
1	(2, 3)	(3,)	(3, 1)	Compare trailing dims: 3 vs 1 (compatible)	Broadcast B to (3,1)	(2, 3) vs (3, 1) shapes not aligned yet
2	(2, 3)	(3, 1)	Broadcast B to (2, 3)	Expand B dims to match A	Add A + B broadcasted	(2, 3)
3	(2, 3)	(3,)	(3, 1)	Final broadcasted shapes: A (2,3), B (2,3)	Element-wise addition	(2, 3)
4	(2, 3)	(3,)	(3, 1)	Result array created	Output C	(2, 3)

💡 Broadcasting completed successfully; result shape is (2, 3)

Variable Tracker

Variable	Start	After Step 1	After Step 2	Final
A	array([[1, 2, 3], [4, 5, 6]])	Same	Same	Same
B	array([10, 20, 30])	Reshaped to (3,1)	Broadcasted to (2,3)	Broadcasted to (2,3)
C	Not defined	Not defined	Computed as A + B broadcasted	array([[11, 12, 13], [24, 25, 26]])

Key Moments - 3 Insights

Why do we reshape B to (3,1) before adding to A?

Reshaping B to (3,1) adds a new axis so its dimensions align properly with A's (2,3) shape for broadcasting, as shown in execution_table step 1.

How does broadcasting handle different shapes like (2,3) and (3,1)?

Broadcasting compares shapes from the right; dimensions of size 1 can be stretched to match the other array, enabling element-wise operations as in execution_table steps 2 and 3.

What happens if shapes are not compatible for broadcasting?

If dimensions differ and neither is 1, broadcasting fails with an error. This is shown in the concept_flow where incompatible shapes lead to an error.

Visual Quiz - 3 Questions

Look at the execution_table at step 1, what is the shape of B after reshaping?

A (3, 1)

B (1, 3)

C (2, 3)

D (3,)

💡 Hint

Check the 'B reshaped shape' column in execution_table row for step 1.

At which step does the final broadcasted shape of B match A's shape?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Look at the 'Broadcasting check' and 'Result shape' columns in execution_table.

If B was not reshaped before addition, what would happen?

A Broadcasting would fail with an error

B B would automatically reshape to (2,3)

C The result would have shape (3,)

D The operation would produce a scalar

💡 Hint

Refer to concept_flow where incompatible shapes cause an error.

Concept Snapshot

Broadcasting with higher dimensions:
- Compare shapes from trailing dimensions
- Dimensions must be equal or one must be 1
- Smaller arrays are reshaped by adding axes
- Arrays expand to compatible shapes
- Element-wise operations then apply
- Result shape is the broadcasted shape

Full Transcript

Broadcasting with higher dimensions means adding arrays of different shapes by expanding the smaller array's dimensions. We compare shapes from the right side. If dimensions match or one is 1, we can stretch the smaller array. For example, adding a (2,3) array to a (3,) array requires reshaping the smaller array to (3,1) first. Then broadcasting expands it to (2,3). The addition happens element-wise, producing a (2,3) result. If shapes are not compatible, broadcasting fails with an error.

Practice

1. What does broadcasting in NumPy allow you to do when working with arrays of different shapes?

easy

A. Copy data from smaller arrays to larger arrays explicitly

B. Perform element-wise operations by stretching smaller arrays to match larger ones

C. Only work with arrays of the exact same shape

D. Convert arrays into lists automatically

Solution

1. Step 1: Understand broadcasting concept

   Broadcasting lets NumPy perform operations on arrays with different shapes by stretching the smaller array without copying data.

2. Step 2: Compare options with concept

   Only Perform element-wise operations by stretching smaller arrays to match larger ones correctly describes this behavior. Options B, C, and D are incorrect because broadcasting does not copy data explicitly, does not require same shape, and does not convert arrays to lists.

3. Final Answer:

   Perform element-wise operations by stretching smaller arrays to match larger ones -> Option B

4. Quick Check:

   Broadcasting = Stretch smaller arrays [OK]

Hint: Broadcasting stretches smaller arrays to match bigger ones [OK]

Common Mistakes:

• Thinking broadcasting copies data
• Believing arrays must have same shape
• Confusing broadcasting with data type conversion

2. Which of the following NumPy array shapes can be broadcast together for element-wise addition?

easy

A. (4, 5) and (4, 5, 1)

B. (2, 3) and (3, 2)

C. (3, 1, 5) and (1, 4, 5)

D. (3, 3) and (2, 3)

Solution

1. Step 1: Check broadcasting rules for each dimension

   Shapes (3,1,5) and (1,4,5) can broadcast because dimensions are compatible: 3 vs 1, 1 vs 4, 5 vs 5.

2. Step 2: Verify other options

   Options A, B, and D have incompatible dimensions that cannot be broadcast for element-wise addition.

3. Final Answer:

   (3, 1, 5) and (1, 4, 5) -> Option C

4. Quick Check:

   Broadcast compatible shapes = (3,1,5) & (1,4,5) [OK]

Hint: Match dimensions from right, 1 or equal size broadcasts [OK]

Common Mistakes:

• Ignoring dimension order when checking compatibility
• Assuming all different shapes can broadcast
• Confusing shape sizes in different axes

3. What is the output of this code?

import numpy as np
A = np.ones((2, 3, 1))
B = np.arange(4).reshape(1, 1, 4)
C = A + B
print(C.shape)

medium

A. ValueError due to shape mismatch

B. (2, 3, 1)

C. (1, 1, 4)

D. (2, 3, 4)

Solution

1. Step 1: Analyze shapes of A and B

   A has shape (2, 3, 1), B has shape (1, 1, 4). Broadcasting rules allow these to combine because 1 can stretch to 2 and 3, and 1 can stretch to 4.

2. Step 2: Determine resulting shape

   The result shape is the maximum along each dimension: max(2,1)=2, max(3,1)=3, max(1,4)=4, so (2, 3, 4).

3. Final Answer:

   (2, 3, 4) -> Option D

4. Quick Check:

   Broadcast result shape = (2,3,4) [OK]

Hint: Result shape = max dimension sizes per axis [OK]

Common Mistakes:

• Confusing which dimension broadcasts to which
• Expecting error instead of broadcasting
• Mixing up shape order

4. Identify the error in this code snippet:

import numpy as np
X = np.ones((3, 1, 2))
Y = np.ones((2, 3))
Z = X + Y

medium

A. Shapes (3,1,2) and (2,3) cannot be broadcast together

B. Y should be reshaped to (1, 2, 3) before addition

C. X and Y have incompatible data types

D. No error, code runs fine

Solution

1. Step 1: Compare shapes for broadcasting

   X shape is (3,1,2), Y shape is (2,3). Aligning from right: 2 vs 3 (mismatch), 1 vs 2 (mismatch), 3 vs missing dimension.

2. Step 2: Conclude broadcasting compatibility

   Dimensions do not match or equal 1, so broadcasting fails and raises an error.

3. Final Answer:

   Shapes (3,1,2) and (2,3) cannot be broadcast together -> Option A

4. Quick Check:

   Broadcast error due to incompatible shapes [OK]

Hint: Check shapes right to left; mismatch means error [OK]

Common Mistakes:

• Assuming automatic reshaping without explicit code
• Ignoring dimension order in broadcasting
• Thinking data types cause this error

5. You have a 3D array data with shape (10, 1, 5) representing 10 samples, 1 channel, and 5 features. You want to add a bias vector of shape (5,) to each sample and channel. Which code correctly applies broadcasting to add the bias to data?

hard

A. bias = np.array([1,2,3,4,5])
data + bias

B. bias = np.array([[1,2,3,4,5]])
data + bias

C. bias = np.array([1,2,3,4,5]).reshape((5,1))
data + bias

D. bias = np.array([[[1,2,3,4,5]]])
data + bias

Solution

1. Step 1: Understand required bias shape and broadcasting

   data shape (10,1,5). Bias vector shape (5,) pads with 1s on left to (1,1,5) and broadcasts correctly across samples and channels.

2. Step 2: Evaluate each option

   A: bias shape (5,), correct. B: shape (1,5) also broadcasts correctly, but is redundant. C: shape (5,1) incompatible. D: shape (1,1,5) also broadcasts correctly, but is more verbose.

3. Final Answer:

   bias = np.array([1,2,3,4,5])
   data + bias -> Option A

4. Quick Check:

   (10,1,5) + (5,) -> broadcasts OK

Hint: Bias shape must match trailing dims with extra dims as 1 [OK]

Common Mistakes:

• Using 1D bias without extra dimensions
• Wrong reshape causing shape mismatch
• Assuming automatic dimension expansion without reshape