Sequence-to-sequence basics in TensorFlow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a sequence-to-sequence (seq2seq) model?

A seq2seq model is a type of neural network that takes a sequence as input and produces another sequence as output. It is often used for tasks like language translation or text summarization.

beginner

What are the two main parts of a seq2seq model?

The two main parts are the encoder and the decoder. The encoder reads the input sequence and creates a summary, and the decoder uses this summary to generate the output sequence.

intermediate

Why do seq2seq models use recurrent neural networks (RNNs) or similar layers?

Because RNNs can process sequences of varying lengths by remembering information from previous steps, making them suitable for handling input and output sequences in seq2seq models.

intermediate

What is the role of the 'context vector' in a basic seq2seq model?

The context vector is the fixed-size summary output by the encoder that contains information about the entire input sequence. The decoder uses it to generate the output sequence.

advanced

What is teacher forcing in seq2seq training?

Teacher forcing is a training technique where the true output from the training data is fed as the next input to the decoder instead of the decoder's own previous prediction. This helps the model learn faster.

What does the encoder in a seq2seq model do?

A Generates the output sequence

B Processes the input sequence and creates a summary

C Calculates the loss function

D Feeds data into the training loop

The encoder reads the input sequence and creates a summary (context vector) for the decoder to use.

Which layer type is commonly used in seq2seq models to handle sequences?

A Convolutional layers

B Pooling layers

C Dense layers only

D Recurrent layers (RNN, LSTM, GRU)

Recurrent layers like LSTM or GRU are used because they can remember information across sequence steps.

What is the purpose of the decoder in a seq2seq model?

A To generate the output sequence from the context vector

B To encode the input sequence

C To preprocess the data

D To calculate accuracy

The decoder uses the context vector from the encoder to produce the output sequence.

What does teacher forcing help with during training?

A Removes the need for a decoder

B Slows down training

C Improves model learning by using true previous outputs

D Increases model size

Teacher forcing feeds the true previous output to the decoder, helping the model learn faster and better.

In a basic seq2seq model, what is the 'context vector'?

A A fixed-size summary of the input sequence

B The output sequence

C The input sequence

D The loss value

The context vector is a fixed-size summary produced by the encoder representing the input sequence.

Explain the roles of the encoder and decoder in a sequence-to-sequence model.

Think about how information flows from input to output.

Describe why recurrent layers like LSTM or GRU are important in seq2seq models.

Consider how the model keeps track of previous words or steps.