Padding and sequence length in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to pad sequences to the same length using Keras.

NLP

from tensorflow.keras.preprocessing.sequence import pad_sequences

sequences = [[1, 2, 3], [4, 5], [6]]
padded = pad_sequences(sequences, maxlen=[1])
print(padded)

Drag options to blanks, or click blank then click option'

A3

B5

C2

D4

Attempts:

3 left

💡 Hint

Common Mistakes

Setting maxlen less than the longest sequence length, which truncates data.

Not using maxlen at all, causing inconsistent sequence lengths.

The longest sequence has length 3, so maxlen=3 pads all sequences to length 3.

2fill in blank

medium

Complete the code to pad sequences with zeros at the end (post-padding).

NLP

from tensorflow.keras.preprocessing.sequence import pad_sequences

sequences = [[7, 8], [9, 10, 11]]
padded = pad_sequences(sequences, padding=[1])
print(padded)

Drag options to blanks, or click blank then click option'

A'post'

B'pre'

C'middle'

D'end'

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'pre' padding which adds zeros before sequences.

Using invalid padding values like 'middle' or 'end'.

Setting padding='post' adds zeros at the end of sequences.

3fill in blank

hard

Fix the error in the code to truncate sequences longer than maxlen from the beginning.

NLP

from tensorflow.keras.preprocessing.sequence import pad_sequences

sequences = [[1, 2, 3, 4], [5, 6, 7]]
padded = pad_sequences(sequences, maxlen=3, truncating=[1])
print(padded)

Drag options to blanks, or click blank then click option'

A'start'

B'post'

C'pre'

D'end'

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'post' truncating which removes elements from the end.

Using invalid truncating values like 'start' or 'end'.

truncating='pre' removes elements from the start to fit maxlen.

4fill in blank

hard

Fill both blanks to create a padded sequence with max length 4, post-padding, and pre-truncating.

NLP

from tensorflow.keras.preprocessing.sequence import pad_sequences

sequences = [[10, 20, 30, 40, 50], [60, 70]]
padded = pad_sequences(sequences, maxlen=[1], padding=[2], truncating='pre')
print(padded)

Drag options to blanks, or click blank then click option'

A4

B'post'

C'pre'

D5

Attempts:

3 left

💡 Hint

Common Mistakes

Using maxlen larger than needed, causing no truncation.

Using 'pre' padding which adds zeros before sequences.

maxlen=4 limits length to 4; padding='post' adds zeros after sequences.

5fill in blank

hard

Fill all three blanks to create a padded sequence with max length 3, pre-padding, and post-truncating.

NLP

from tensorflow.keras.preprocessing.sequence import pad_sequences

sequences = [[100, 200, 300, 400], [500]]
padded = pad_sequences(sequences, maxlen=[1], padding=[2], truncating=[3])
print(padded)

Drag options to blanks, or click blank then click option'

A3

B'pre'

C'post'

D'start'

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'post' padding which adds zeros after sequences.

Using 'pre' truncating which removes from the start.

maxlen=3 limits length; padding='pre' adds zeros before; truncating='post' removes from end.

Practice

1. What is the main purpose of padding in text sequences for machine learning models?

easy

A. To convert text into numbers without changing length

B. To make all sequences the same length by adding extra values

C. To randomly shuffle the words in sequences

D. To remove important words from sequences

Solution

1. Step 1: Understand padding concept

   Padding adds extra values (usually zeros) to sequences to make them all the same length.

2. Step 2: Recognize why padding is used

   This uniform length helps models process batches of data efficiently without errors.

3. Final Answer:

   To make all sequences the same length by adding extra values -> Option B

4. Quick Check:

   Padding = same length sequences [OK]

Hint: Padding adds extra tokens to equalize sequence lengths [OK]

Common Mistakes:

• Thinking padding removes words
• Confusing padding with shuffling
• Believing padding changes text meaning

2. Which of the following is the correct way to pad sequences using Python's Keras library?

easy

A. pad_sequences(sequences, maxlen=10, shuffle=True)

B. pad_sequences(sequences, maxlen=10, reverse=True)

C. pad_sequences(sequences, maxlen=10, padding='post')

D. pad_sequences(sequences, maxlen=10, drop=True)

Solution

1. Step 1: Identify correct padding function parameters

   Keras's pad_sequences uses 'padding' to specify where to add zeros, e.g., 'post' means after the sequence.

2. Step 2: Check options for valid parameters

   Only pad_sequences(sequences, maxlen=10, padding='post') uses a valid parameter 'padding' with a correct value 'post'. Others use invalid parameters like shuffle, reverse, drop.

3. Final Answer:

   pad_sequences(sequences, maxlen=10, padding='post') -> Option C

4. Quick Check:

   Correct padding param = pad_sequences(sequences, maxlen=10, padding='post') [OK]

Hint: Use 'padding' param in pad_sequences, not shuffle or drop [OK]

Common Mistakes:

• Using non-existent parameters like shuffle or drop
• Confusing padding location with sequence order
• Forgetting to set maxlen for fixed length

3. Given the code below, what will be the output shape of padded_sequences?

from tensorflow.keras.preprocessing.sequence import pad_sequences
sequences = [[1, 2, 3], [4, 5], [6]]
padded_sequences = pad_sequences(sequences, maxlen=4, padding='pre')

medium

A. (3, 4)

B. (4, 3)

C. (3, 3)

D. (4, 4)

Solution

1. Step 1: Count number of sequences

   There are 3 sequences: [1,2,3], [4,5], and [6].

2. Step 2: Understand padding effect on length

   maxlen=4 means each sequence is padded or truncated to length 4. So output shape is (3 sequences, 4 length each).

3. Final Answer:

   (3, 4) -> Option A

4. Quick Check:

   Number sequences = 3, length = 4 [OK]

Hint: Output shape = (number sequences, maxlen) [OK]

Common Mistakes:

• Confusing maxlen with number of sequences
• Mixing up padding='pre' with output shape
• Assuming shape changes with padding side

4. You wrote this code but get an error: TypeError: pad_sequences() got an unexpected keyword argument 'pad'. What is the likely mistake?

padded = pad_sequences(sequences, maxlen=5, pad='post')

medium

A. The parameter name should be 'padding', not 'pad'

B. maxlen must be smaller than sequence length

C. Sequences must be numpy arrays, not lists

D. pad_sequences requires a 'value' parameter

Solution

1. Step 1: Identify error cause from message

   The error says 'unexpected keyword argument pad', meaning 'pad' is not a valid parameter.

2. Step 2: Recall correct parameter name

   The correct parameter to specify padding side is 'padding', not 'pad'.

3. Final Answer:

   The parameter name should be 'padding', not 'pad' -> Option A

4. Quick Check:

   Correct param = 'padding' [OK]

Hint: Use 'padding' param, not 'pad' [OK]

Common Mistakes:

• Using 'pad' instead of 'padding'
• Assuming maxlen must be smaller than sequences
• Thinking sequences must be numpy arrays

5. You have text sequences of varying lengths. You want to pad them to length 10 but keep the last 10 words only if longer. Which code correctly achieves this using Keras?

hard

A. pad_sequences(sequences, maxlen=10, padding='post', truncating='pre')

B. pad_sequences(sequences, maxlen=10, padding='post', truncating='post')

C. pad_sequences(sequences, maxlen=10, padding='pre', truncating='post')

D. pad_sequences(sequences, maxlen=10, padding='pre', truncating='pre')

Solution

1. Step 1: Understand padding and truncating sides

   Padding='pre' adds zeros at the start; truncating='pre' removes words from the start, keeping last words.

2. Step 2: Match requirement to keep last 10 words

   To keep last 10 words, truncate from the start ('pre') and pad at the start ('pre').

3. Final Answer:

   pad_sequences(sequences, maxlen=10, padding='pre', truncating='pre') -> Option D

4. Quick Check:

   Keep last words = truncating='pre' [OK]

Hint: Use truncating='pre' to keep last words, padding='pre' to pad start [OK]

Common Mistakes:

• Using padding='post' which pads end instead of start
• Using truncating='post' which removes last words
• Confusing padding and truncating parameters