Sequence-to-sequence basics in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of a sequence-to-sequence model?

In simple terms, what does a sequence-to-sequence model do?

A It predicts a single number from a fixed set of input features.

B It converts one sequence of data into another sequence, like translating a sentence from one language to another.

C It classifies images into categories based on their content.

D It clusters data points into groups without labels.

💡 Hint

Think about tasks like language translation or summarizing text.

Sequence-to-sequence models take a sequence as input and produce a sequence as output, useful for tasks like translation or speech recognition.

❓ Predict Output

intermediate

Output shape of encoder LSTM in a seq2seq model

Given the following TensorFlow code for an encoder LSTM, what is the shape of the output 'encoder_outputs'?

TensorFlow

import tensorflow as tf
encoder_inputs = tf.keras.Input(shape=(None, 10))
encoder_lstm = tf.keras.layers.LSTM(16, return_sequences=True, return_state=True)
encoder_outputs, state_h, state_c = encoder_lstm(encoder_inputs)

A (None, None, 16)

B (None, 16)

C (None, 10, 16)

D (None, 16, 16)

💡 Hint

return_sequences=True means output has one vector per input time step.

With return_sequences=True, the output shape is (batch_size, timesteps, units). Here units=16 and timesteps is variable (None).

❓ Model Choice

advanced

Choosing the right decoder output activation for sequence generation

You are building a sequence-to-sequence model to generate text word-by-word from a vocabulary of 5000 words. Which decoder output activation function is most appropriate?

A Softmax activation to output probabilities over the vocabulary

B Linear activation to output raw scores without normalization

C ReLU activation to output positive scores for each word

D Sigmoid activation to output independent probabilities for each word

💡 Hint

Think about how to pick one word from many possible words at each step.

Softmax outputs a probability distribution over all words, allowing selection of the most likely next word.

❓ Hyperparameter

advanced

Effect of increasing LSTM units in a seq2seq model

What is the most likely effect of increasing the number of units in the LSTM layers of a sequence-to-sequence model?

A The model will ignore input sequences and output random predictions

B The model will train faster and use less memory

C The model will always perform worse due to overfitting

D The model can capture more complex patterns but may overfit if data is limited

💡 Hint

More units mean more capacity but also more risk of memorizing training data.

Increasing units increases model capacity, helping learn complex patterns but risking overfitting without enough data.

❓ Metrics

expert

Choosing the best metric for evaluating a seq2seq translation model

You trained a sequence-to-sequence model for language translation. Which metric best measures how close the model's output sentences are to the correct translations?

A Accuracy, counting exact matches of entire output sequences

B Mean Squared Error, measuring average squared difference between predicted and true word indices

C BLEU score, which compares overlapping n-grams between output and reference sentences

D Perplexity, measuring how well the model predicts the next word probabilities

💡 Hint

Think about a metric that compares similarity of sentences rather than exact matches.

BLEU score measures n-gram overlap, capturing partial matches and fluency in translation quality.