Padding and sequence length in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why do we pad sequences in NLP models?

In natural language processing, why is padding used when preparing sequences for models?

A To make all sequences the same length so they can be processed in batches.

B To convert text into numerical values.

C To increase the vocabulary size of the model.

D To remove stop words from the sequences.

💡 Hint

Think about how models handle input data in batches.

Padding makes all sequences the same length, allowing batch processing. Without padding, sequences of different lengths can't be processed together efficiently.

❓ Predict Output

intermediate

Output length after padding sequences

Given the following code that pads sequences to a max length of 5, what is the output?

NLP

from tensorflow.keras.preprocessing.sequence import pad_sequences
sequences = [[1, 2, 3], [4, 5], [6]]
padded = pad_sequences(sequences, maxlen=5, padding='post')
print(padded.tolist())

A [[1, 2, 3], [4, 5], [6]]

B [[0, 0, 1, 2, 3], [0, 0, 0, 4, 5], [0, 0, 0, 0, 6]]

C [[1, 2, 3, 0, 0], [4, 5, 0, 0, 0], [6, 0, 0, 0, 0]]

D [[1, 2, 3, 0], [4, 5, 0], [6, 0, 0]]

💡 Hint

Check the padding='post' argument and maxlen=5.

Padding='post' adds zeros after the sequence to reach length 5.

❓ Model Choice

advanced

Choosing sequence length for RNN training

You have text sequences of varying lengths from 10 to 100 tokens. You want to train an RNN model efficiently. Which sequence length choice is best?

A Pad all sequences to length 100 to keep full information.

B Pad sequences to the median length around 50 and truncate longer ones.

C Pad all sequences to length 10 to speed up training.

D Do not pad sequences and train with variable lengths.

💡 Hint

Consider balancing information retention and training speed.

Padding to median length balances keeping enough information and efficient training. Padding to max length wastes resources; too short loses info; variable lengths complicate batching.

❓ Metrics

advanced

Effect of padding on model accuracy

When training a text classification model, how can excessive padding affect accuracy?

A Padding causes the model to overfit the training data.

B Padding has no effect on accuracy since it is ignored by the model.

C More padding always improves accuracy by standardizing input size.

D Excessive padding can introduce noise and reduce accuracy.

💡 Hint

Think about how meaningless zeros might affect learning.

Excessive padding adds many zeros that do not carry information, which can confuse the model and reduce accuracy.

🔧 Debug

expert

Why does this padding code truncate unexpectedly?

Consider this code snippet:

from tensorflow.keras.preprocessing.sequence import pad_sequences
sequences = [[1, 2], [3, 4, 5, 6]]
padded = pad_sequences(sequences, maxlen=3, padding='post')
print(padded)

Why does this code truncate unexpectedly?

A Because maxlen is smaller than the length of one sequence, causing truncation without specifying truncating='post'.

B Because pad_sequences requires all sequences to be the same length before padding.

C Because sequences contain integers instead of strings.

D Because padding='post' is invalid and should be 'pre'.

💡 Hint

Check the maxlen parameter and sequence lengths.

When maxlen is smaller than some sequences, pad_sequences truncates them. If truncating parameter is not set, it defaults to 'pre'. This can cause unexpected behavior if sequences are longer than maxlen.