Indexing and slicing tensors in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Indexing and slicing tensors

Problem:You want to extract specific parts of a tensor to use in your model or analysis. Currently, you do not know how to correctly index or slice tensors in TensorFlow.

Current Metrics:No metrics yet, as this is about data manipulation before modeling.

Issue:Without proper indexing and slicing, you cannot efficiently select or manipulate tensor data, which is essential for preparing inputs or inspecting outputs.

Your Task

Learn to extract specific elements, rows, columns, and sub-tensors from a given 3D tensor using TensorFlow indexing and slicing.

Use TensorFlow 2.x API only.

Do not convert tensors to numpy arrays for slicing.

Demonstrate at least four different types of indexing/slicing.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

TensorFlow

import tensorflow as tf

# Create a 3D tensor of shape (3, 4, 5)
tensor = tf.constant([[[i + j + k for k in range(5)] for j in range(4)] for i in range(3)])

# 1. Indexing: Get the first element of the tensor (shape (4,5))
first_element = tensor[0]

# 2. Slicing: Get the first two rows of the first element (shape (2,5))
slice_rows = tensor[0, 0:2, :]

# 3. Indexing: Get the element at position [2,3,4]
single_value = tensor[2, 3, 4]

# 4. Using tf.slice: Extract a sub-tensor starting at [1,1,1] with size [2,2,3]
sub_tensor = tf.slice(tensor, begin=[1, 1, 1], size=[2, 2, 3])

# Print results
print('Original tensor shape:', tensor.shape)
print('First element (tensor[0]) shape:', first_element.shape)
print('Slice rows (tensor[0, 0:2, :]) shape:', slice_rows.shape)
print('Single value (tensor[2, 3, 4]):', single_value.numpy())
print('Sub-tensor (tf.slice):', sub_tensor.numpy())

Created a 3D tensor using tf.constant.

Used direct indexing to get the first element.

Used slicing to get a subset of rows and all columns.

Used tf.slice to extract a sub-tensor with specific start and size.

Results Interpretation

Before: No ability to select parts of tensors, limiting data manipulation.

After: Able to extract specific elements, slices, and sub-tensors efficiently using TensorFlow indexing and slicing.

Mastering indexing and slicing in TensorFlow lets you handle tensor data flexibly, which is crucial for preparing inputs and analyzing outputs in machine learning.

Bonus Experiment

Try to extract a diagonal slice from a 3D tensor and explain how you did it.

💡 Hint

Use tf.linalg.diag_part or combine tf.range with tf.gather_nd for advanced indexing.

Practice

1. What does indexing a tensor in TensorFlow do?

easy

A. Selects a single element by its position

B. Changes the shape of the tensor

C. Adds new elements to the tensor

D. Deletes elements from the tensor

Solution

1. Step 1: Understand indexing

   Indexing means picking one element from a tensor by its position, like choosing one item from a list.

2. Step 2: Compare with other options

   Changing shape, adding, or deleting elements are different operations, not indexing.

3. Final Answer:

   Selects a single element by its position -> Option A

4. Quick Check:

   Indexing = single element pick [OK]

Hint: Indexing picks one element, slicing picks many [OK]

Common Mistakes:

• Thinking indexing changes tensor shape
• Confusing indexing with adding elements
• Assuming indexing deletes elements

2. Which of the following is the correct syntax to slice a 1D tensor t from index 2 to 5 (exclusive) in TensorFlow?

easy

A. t[2:5]

B. t.slice(2, 5)

C. t[2, 5]

D. t.slice(2:5)

Solution

1. Step 1: Recall slicing syntax

   TensorFlow uses Python-style slicing: t[start:stop] to get elements from start up to but not including stop.

2. Step 2: Check each option

   t[2:5] uses correct Python slice syntax. t.slice(2, 5) and D use incorrect method calls or syntax. t[2, 5] uses comma which is invalid for 1D slicing.

3. Final Answer:

   t[2:5] -> Option A

4. Quick Check:

   Slice syntax = t[start:stop] [OK]

Hint: Use square brackets with colon for slicing [OK]

Common Mistakes:

• Using commas instead of colons in slices
• Trying to call slice as a method incorrectly
• Confusing slice stop index as inclusive

3. Given the tensor t = tf.constant([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), what is the output of t[1:, :2].numpy()?

medium

A. [[5 6]
[8 9]]

B. [[1 2]
[4 5]]

C. [[4 5 6]
[7 8 9]]

D. [[4 5]
[7 8]]

Solution

1. Step 1: Understand slicing t[1:, :2]

   1: means rows from index 1 to end (rows 1 and 2). :2 means columns from start to index 2 (columns 0 and 1).

2. Step 2: Extract the sliced elements

   Rows 1 and 2 are [[4,5,6], [7,8,9]]. Taking first two columns gives [[4,5], [7,8]].

3. Final Answer:

   [[4 5] [7 8]] -> Option D

4. Quick Check:

   Rows 1+ and cols 0-1 = [[4 5],[7 8]] [OK]

Hint: Remember slice stop is exclusive, so :2 means columns 0 and 1 [OK]

Common Mistakes:

• Including column index 2 mistakenly
• Starting rows from 0 instead of 1
• Confusing rows and columns order

4. What is wrong with this TensorFlow slicing code?

t = tf.constant([10, 20, 30, 40, 50])
slice = t[1:6]

medium

A. Index 6 is out of range, causing an error

B. Slicing with stop index beyond length is allowed, no error

C. Syntax error due to missing colon

D. TensorFlow does not support slicing

Solution

1. Step 1: Check slicing behavior with stop index

   In Python and TensorFlow, slicing stop index can be beyond tensor length without error; it stops at the end.

2. Step 2: Analyze given code

   Tensor t has length 5, slicing 1:6 extracts elements from index 1 to end safely.

3. Final Answer:

   Slicing with stop index beyond length is allowed, no error -> Option B

4. Quick Check:

   Slice stop > length is safe [OK]

Hint: Slice stop can exceed length without error [OK]

Common Mistakes:

• Expecting IndexError for slice stop beyond length
• Confusing slicing with indexing single element
• Thinking slicing syntax is invalid

5. You have a 3D tensor t = tf.constant([[[1,2],[3,4]], [[5,6],[7,8]], [[9,10],[11,12]]]). How do you extract the second element from each 2D matrix (i.e., elements 2, 4, 6, 8, 10, 12) using indexing and slicing?

hard

A. t[1, :, :]

B. t[:, 1, :]

C. t[:, :, 1]

D. t[:, 1]

Solution

1. Step 1: Understand tensor shape and indexing

   The tensor shape is (3, 2, 2): 3 matrices, each 2x2. We want the second element in the last dimension (index 1).

2. Step 2: Apply slicing to get second element in last dimension

   Using t[:, :, 1] selects all matrices (:), all rows (:), and the second element (index 1) in the last dimension.

3. Final Answer:

   t[:, :, 1] -> Option C

4. Quick Check:

   Last dim index 1 selects second elements [OK]

Hint: Use colon for all dims except last, index last dim 1 [OK]

Common Mistakes:

• Mixing row and column indices
• Using incomplete slicing like t[:, 1]
• Selecting wrong dimension index