Tensor shapes and dimensions in PyTorch - ML Experiment: Train & Evaluate

Experiment - Tensor shapes and dimensions

Problem:You are working with tensors in PyTorch and want to understand how their shapes and dimensions affect operations like matrix multiplication and reshaping.

Current Metrics:No model training metrics yet; the current challenge is understanding tensor shapes and how to manipulate them correctly.

Issue:You often get errors due to mismatched tensor shapes when performing operations like matrix multiplication or reshaping tensors incorrectly.

Your Task

Learn to correctly identify tensor shapes and dimensions, and perform basic tensor operations such as reshaping and matrix multiplication without errors.

Use PyTorch tensors only.

Do not use any external libraries for tensor operations.

Keep tensor dimensions compatible for operations.

Hint 1

Hint 2

Hint 3

Solution

PyTorch

import torch

# Create two tensors
# Tensor A: shape (2, 3)
A = torch.tensor([[1, 2, 3], [4, 5, 6]])
# Tensor B: shape (3, 2)
B = torch.tensor([[7, 8], [9, 10], [11, 12]])

print(f"Shape of A: {A.shape}")  # Output: torch.Size([2, 3])
print(f"Shape of B: {B.shape}")  # Output: torch.Size([3, 2])

# Matrix multiplication: (2,3) x (3,2) -> (2,2)
C = torch.matmul(A, B)
print(f"Result of A x B: {C}")
print(f"Shape of C: {C.shape}")  # Output: torch.Size([2, 2])

# Reshape tensor C from (2, 2) to (4,)
C_reshaped = C.view(4)
print(f"Reshaped C: {C_reshaped}")
print(f"Shape of reshaped C: {C_reshaped.shape}")

# Try an incorrect reshape (will raise error if uncommented)
# C.view(3)  # Cannot reshape 4 elements into shape (3,)

# Create a tensor D with shape (4, 1, 2)
D = torch.arange(8).view(4, 1, 2)
print(f"Tensor D: {D}")
print(f"Shape of D: {D.shape}")

# Squeeze removes dimensions of size 1
D_squeezed = D.squeeze()
print(f"Shape of D after squeeze: {D_squeezed.shape}")  # Output: torch.Size([4, 2])

# Unsqueeze adds a dimension at position 1
D_unsqueezed = D_squeezed.unsqueeze(1)
print(f"Shape of D after unsqueeze: {D_unsqueezed.shape}")  # Output: torch.Size([4, 1, 2])

Added tensor creation with specific shapes to demonstrate shape properties.

Performed matrix multiplication with compatible shapes to show correct operation.

Used view() to reshape tensors correctly and showed error example for incorrect reshape.

Demonstrated squeeze() and unsqueeze() to manipulate tensor dimensions.

Results Interpretation

Before: Frequent errors due to shape mismatches and confusion about tensor dimensions.
After: Clear understanding of tensor shapes, successful matrix multiplication, and reshaping without errors.

Understanding tensor shapes and dimensions is crucial for performing operations correctly in PyTorch. Checking shapes and using reshape functions carefully prevents errors and enables flexible tensor manipulation.

Bonus Experiment

Try performing batch matrix multiplication with tensors of shape (batch_size, n, m) and (batch_size, m, p).

💡 Hint

Use torch.bmm() for batch matrix multiplication and ensure batch sizes and inner dimensions match.