Indexing and slicing in PyTorch - Model Pipeline Trace

Model Pipeline - Indexing and slicing

This pipeline shows how data tensors are accessed and manipulated using indexing and slicing in PyTorch. It helps select parts of data for further processing or model input.

Data Flow - 4 Stages

1Input tensor

1 tensor of shape (4, 5)→Create a 2D tensor with 4 rows and 5 columns→1 tensor of shape (4, 5)

[[10, 20, 30, 40, 50], [60, 70, 80, 90, 100], [110, 120, 130, 140, 150], [160, 170, 180, 190, 200]]

↓

2Row indexing

1 tensor of shape (4, 5)→Select the 2nd row using indexing→1 tensor of shape (5,)

[60, 70, 80, 90, 100]

↓

3Column slicing

1 tensor of shape (4, 5)→Select columns 1 to 3 (exclusive) for all rows using slicing→1 tensor of shape (4, 2)

[[20, 30], [70, 80], [120, 130], [170, 180]]

↓

4Combined indexing and slicing

1 tensor of shape (4, 5)→Select the last two columns of the 3rd row→1 tensor of shape (2,)

[140, 150]

Training Trace - Epoch by Epoch

Loss
1.0 | *       
0.8 |  *      
0.6 |   *     
0.4 |    *    
0.2 |     *   
0.0 +---------
      1 2 3 4 5
      Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.40	Initial loss is high; accuracy is low as model starts learning.
2	0.65	0.55	Loss decreases and accuracy improves after first update.
3	0.50	0.70	Model learns important features; metrics improve steadily.
4	0.38	0.80	Loss continues to drop; accuracy reaches a good level.
5	0.30	0.85	Training converges with low loss and high accuracy.

Prediction Trace - 4 Layers

Layer 1: Input tensor

Layer 2: Row indexing

Layer 3: Column slicing

Layer 4: Combined indexing and slicing

Model Quiz - 3 Questions

What is the shape of the tensor after selecting the 2nd row from a (4, 5) tensor?

A (1, 5)

B (4,)

C (5,)

D (4, 5)

Key Insight

Indexing and slicing let us pick specific parts of data tensors easily. This is useful to focus on relevant features or samples before feeding data into a model. Understanding shapes after these operations helps avoid errors and ensures correct data flow.

Practice

1. What does the expression tensor[2] do in PyTorch when tensor is a 1D tensor?

easy

A. Selects every second element starting at index 2

B. Selects elements from index 0 to 2

C. Selects the element at index 2

D. Raises an error because indexing starts at 1

Solution

1. Step 1: Understand 1D tensor indexing

   In PyTorch, indexing a 1D tensor with [2] picks the element at position 2 (0-based index).

2. Step 2: Confirm indexing behavior

   Since indexing starts at 0, tensor[2] returns the third element.

3. Final Answer:

   Selects the element at index 2 -> Option C

4. Quick Check:

   Indexing with a single integer picks one element [OK]

Hint: Single index picks one element by position [OK]

Common Mistakes:

• Thinking indexing starts at 1
• Confusing slicing with indexing
• Expecting a range instead of one element

2. Which of the following is the correct syntax to slice a 1D tensor t to get elements from index 1 to 4 (excluding 4)?

easy

A. t(1:4)

B. t[1:4]

C. t[1-4]

D. t[1,4]

Solution

1. Step 1: Recall slicing syntax

   Slicing uses start:stop inside square brackets to select elements from start index up to but not including stop index.

2. Step 2: Check each option

   t[1:4] uses correct slicing syntax t[1:4]. Options A, B, and C use invalid syntax for slicing.

3. Final Answer:

   t[1:4] -> Option B

4. Quick Check:

   Slicing uses colon inside brackets [OK]

Hint: Use colon inside brackets for slicing [OK]

Common Mistakes:

• Using comma instead of colon for slicing
• Using parentheses instead of brackets
• Using minus sign instead of colon

3. Given the tensor t = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), what is the output of t[1:, :2]?

medium

A. [[5, 6], [8, 9]]

B. [[2, 3], [5, 6]]

C. [[1, 2], [4, 5]]

D. [[4, 5], [7, 8]]

Solution

1. Step 1: Understand slicing t[1:, :2]

   1: means rows from index 1 to end (rows 1 and 2). :2 means columns from start to index 2 (columns 0 and 1).

2. Step 2: Extract the sliced elements

   Rows 1 and 2 are [[4,5,6],[7,8,9]]. Taking columns 0 and 1 gives [[4,5],[7,8]].

3. Final Answer:

   [[4, 5], [7, 8]] -> Option D

4. Quick Check:

   Row slice 1: and col slice :2 gives [[4,5],[7,8]] [OK]

Hint: Rows after 1, columns before 2 slice tensor [OK]

Common Mistakes:

• Mixing row and column indices
• Including column index 2 mistakenly
• Confusing slicing ranges

4. What is wrong with this code snippet?

import torch
t = torch.tensor([1, 2, 3, 4, 5])
print(t[::2, 1])

medium

A. Trying to index 2D on a 1D tensor

B. Slicing syntax is incorrect

C. Tensor is not created properly

D. Print statement missing parentheses

Solution

1. Step 1: Analyze tensor shape

   The tensor t is 1D with shape (5,). It has only one dimension.

2. Step 2: Check indexing used

   The code uses t[::2, 1] which tries to index two dimensions (rows and columns). This causes an error because t is 1D.

3. Final Answer:

   Trying to index 2D on a 1D tensor -> Option A

4. Quick Check:

   1D tensor cannot be indexed with two indices [OK]

Hint: Match number of indices to tensor dimensions [OK]

Common Mistakes:

• Assuming 1D tensor can be indexed like 2D
• Confusing slicing syntax with indexing
• Ignoring tensor shape before indexing

5. You have a 3D tensor t with shape (4, 5, 6). Which indexing expression extracts all elements from the second dimension at index 3, across all batches and features?

hard

A. t[:, 3, :]

B. t[3, :, :]

C. t[:, :, 3]

D. t[3, 3, :]

Solution

1. Step 1: Understand tensor dimensions

   Tensor shape is (4, 5, 6): first dim = batches, second dim = 5 elements, third dim = 6 features.

2. Step 2: Select all batches, index 3 in second dim, all features

   Use : to select all in first and third dims, and 3 to select index 3 in second dim, so t[:, 3, :].

3. Final Answer:

   t[:, 3, :] -> Option A

4. Quick Check:

   Colon means all, index 3 picks second dim element [OK]

Hint: Use colon for all dims, index for specific dim [OK]

Common Mistakes:

• Mixing dimension order
• Using wrong index for dimension
• Using single index for multiple dims