Text preprocessing for RNNs in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to tokenize the input sentence into words.

PyTorch

sentence = "I love machine learning"
tokens = sentence.[1]()

Drag options to blanks, or click blank then click option'

Asplit

Bjoin

Creplace

Dstrip

Attempts:

3 left

💡 Hint

Common Mistakes

Using join() instead of split() which combines words instead of splitting.

Using replace() which changes characters but does not split.

Using strip() which removes whitespace only at ends.

The split() method breaks the sentence into a list of words by spaces.

2fill in blank

medium

Complete the code to convert tokens to lowercase for uniformity.

PyTorch

tokens = ['I', 'Love', 'Machine', 'Learning']
lower_tokens = [word.[1]() for word in tokens]

Drag options to blanks, or click blank then click option'

Aupper

Blower

Ccapitalize

Dtitle

Attempts:

3 left

💡 Hint

Common Mistakes

Using upper() which converts to uppercase instead.

Using capitalize() which only changes the first letter.

Using title() which capitalizes each word.

The lower() method converts each word to lowercase, which helps the model treat words like 'Love' and 'love' the same.

3fill in blank

hard

Fix the error in the code to create a vocabulary dictionary mapping words to unique indices.

PyTorch

tokens = ['i', 'love', 'machine', 'learning', 'love']
vocab = {word: idx for idx, word in enumerate(set([1]))}

Drag options to blanks, or click blank then click option'

Alist

Bvocab

Ctokens

Drange

Attempts:

3 left

💡 Hint

Common Mistakes

Using vocab inside set() which is not defined yet.

Using list which is a type, not the variable.

Using range which is unrelated here.

We use set(tokens) to get unique words, then enumerate to assign each a unique index.

4fill in blank

hard

Fill both blanks to convert a list of tokens into a list of indices using the vocabulary.

PyTorch

tokens = ['i', 'love', 'machine']
indices = [[1][word] for [2] in tokens]

Drag options to blanks, or click blank then click option'

Avocab

Bword

Ctokens

Dindices

Attempts:

3 left

💡 Hint

Common Mistakes

Using tokens instead of vocab to get indices.

Using tokens as loop variable which is the list, not element.

Using indices as loop variable which is the output list.

We use the vocabulary dictionary vocab to get the index for each word in the tokens list.

5fill in blank

hard

Fill all three blanks to pad sequences to the same length using PyTorch.

PyTorch

import torch
from torch.nn.utils.rnn import [1]

sequences = [torch.tensor([1, 2, 3]), torch.tensor([4, 5])]
padded = [2](sequences, batch_first=True, padding_value=[3])

Drag options to blanks, or click blank then click option'

Apad_sequence

Bpad_packed_sequence

Cpack_sequence

D0

Attempts:

3 left

💡 Hint

Common Mistakes

Using pack_sequence which packs sequences but does not pad.

Using pad_packed_sequence which unpacks padded sequences.

Using pack_padded_sequence which packs padded sequences.

pad_sequence pads a list of variable-length tensors to the same length. We use padding_value=0 to pad with zeros.

Practice

1. Why do we split text into tokens before feeding it to an RNN?

easy

A. Because RNNs process sequences of numbers, not raw text

B. To reduce the size of the dataset

C. To make the text look nicer

D. Because tokens are easier to print

Solution

1. Step 1: Understand RNN input requirements

   RNNs work with sequences of numbers, not raw text strings.

2. Step 2: Role of tokenization

   Splitting text into tokens converts sentences into smaller units that can be mapped to numbers.

3. Final Answer:

   Because RNNs process sequences of numbers, not raw text -> Option A

4. Quick Check:

   Tokenization = Convert text to numbers [OK]

Hint: RNNs need numbers, so split text into tokens first [OK]

Common Mistakes:

• Thinking tokens are for making text prettier
• Believing tokenization reduces dataset size
• Confusing tokens with characters

2. Which PyTorch function is commonly used to pad sequences to the same length for batch processing?

easy

A. torch.nn.utils.rnn.pad_sequence

B. torch.tensor.pad

C. torch.pad_sequences

D. torch.nn.pad

Solution

1. Step 1: Identify PyTorch padding utilities

   PyTorch provides pad_sequence in torch.nn.utils.rnn to pad variable-length sequences.

2. Step 2: Check other options

   Functions like torch.tensor.pad or torch.nn.pad do not exist; torch.pad_sequences is not a PyTorch function.

3. Final Answer:

   torch.nn.utils.rnn.pad_sequence -> Option A

4. Quick Check:

   Use pad_sequence to pad RNN inputs [OK]

Hint: Remember: pad_sequence is in torch.nn.utils.rnn [OK]

Common Mistakes:

• Using non-existent torch.pad_sequences
• Confusing tensor.pad with pad_sequence
• Trying to pad manually without this function

3. Given the following code, what is the shape of the padded batch tensor?

import torch
from torch.nn.utils.rnn import pad_sequence

seq1 = torch.tensor([1, 2, 3])
seq2 = torch.tensor([4, 5])
seq3 = torch.tensor([6])
batch = pad_sequence([seq1, seq2, seq3], batch_first=True, padding_value=0)
print(batch.shape)

medium

A. (1, 3)

B. (3, 1)

C. (3, 3)

D. (3, 6)

Solution

1. Step 1: Understand input sequences

   Sequences have lengths 3, 2, and 1 respectively.

2. Step 2: pad_sequence with batch_first=true

   All sequences are padded to length 3 (max length), batch dimension is first, so shape is (3 sequences, 3 elements each).

3. Final Answer:

   (3, 3) -> Option C

4. Quick Check:

   Batch size = 3, max seq length = 3 [OK]

Hint: Batch shape = (number sequences, max sequence length) [OK]

Common Mistakes:

• Confusing batch_first=true with false
• Assuming padding adds length beyond max sequence
• Mixing up batch and sequence dimensions

4. What is wrong with this code snippet for preparing text sequences for an RNN?

import torch
from torch.nn.utils.rnn import pad_sequence

sentences = [[1, 2, 3, 4], [5, 6], [7]]
tensors = [torch.tensor(s) for s in sentences]
padded = pad_sequence(tensors)
print(padded.shape)

medium

A. torch.tensor cannot convert lists to tensors

B. pad_sequence is missing batch_first=true, so shape is unexpected

C. pad_sequence requires padding_value argument

D. The input lists must be numpy arrays, not lists

Solution

1. Step 1: Check pad_sequence default behavior

   By default, pad_sequence returns tensor with shape (max_seq_len, batch_size), not batch first.

2. Step 2: Effect on output shape

   Without batch_first=true, the printed shape will be (4, 3) instead of expected batch-first (3, 4) shape.

3. Final Answer:

   pad_sequence is missing batch_first=true, so shape is unexpected -> Option B

4. Quick Check:

   Use batch_first=true for (batch, seq_len) shape [OK]

Hint: Always add batch_first=true for batch as first dimension [OK]

Common Mistakes:

• Assuming pad_sequence pads automatically without batch_first
• Thinking torch.tensor can't convert lists
• Believing padding_value is mandatory

5. You have a batch of sentences tokenized as integer lists of different lengths. You want to feed them into an RNN in PyTorch. Which sequence of steps is correct for preprocessing?

hard

A. Tokenize text -> Pad sequences -> Convert tokens to integers -> Feed to RNN

B. Pad raw text strings -> Tokenize padded strings -> Convert tokens to integers -> Feed to RNN

C. Convert raw text to tensor -> Tokenize tensor -> Pad sequences -> Feed to RNN

D. Tokenize text -> Convert tokens to integers -> Pad sequences with pad_sequence(batch_first=true) -> Convert to tensor batch

Solution

1. Step 1: Tokenize text and convert tokens to integers

   First, split text into tokens, then map tokens to integers using a vocabulary.

2. Step 2: Pad sequences and prepare batch tensor

   Pad integer sequences to equal length using pad_sequence with batch_first=true, then feed the tensor batch to the RNN.

3. Final Answer:

   Tokenize text -> Convert tokens to integers -> Pad sequences with pad_sequence(batch_first=true) -> Convert to tensor batch -> Option D

4. Quick Check:

   Tokenize -> Integer map -> Pad -> Batch tensor [OK]

Hint: Tokenize first, then integer map, then pad sequences [OK]

Common Mistakes:

• Padding raw text instead of token integers
• Converting raw text directly to tensor
• Padding before converting tokens to integers