Transformer architecture overview in Prompt Engineering / GenAI - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main purpose of the Transformer architecture in AI?

The Transformer architecture is designed to process sequences of data, like sentences, by focusing on relationships between all parts of the sequence at once, enabling better understanding and generation of language.

beginner

What does 'self-attention' mean in the Transformer model?

Self-attention is a mechanism where the model looks at all words in a sentence to decide which words are important to understand each word better, helping it capture context effectively.

beginner

Name the two main parts of the Transformer architecture.

The Transformer has two main parts: the Encoder, which reads and understands the input data, and the Decoder, which generates the output based on the Encoder's understanding.

intermediate

Why does the Transformer use 'positional encoding'?

Because Transformers process all words at once, positional encoding adds information about the order of words so the model knows the sequence in which words appear.

intermediate

How does the Transformer differ from older sequence models like RNNs?

Unlike RNNs that process words one by one, Transformers look at all words simultaneously using self-attention, which allows faster training and better understanding of long-range relationships.

What is the role of the Encoder in a Transformer?

A To generate the output sequence

B To read and understand the input data

C To add positional information

D To perform self-attention only on output

The Encoder reads and processes the input data to create a representation that the Decoder can use.

What does self-attention help the Transformer model do?

A Ignore word order

B Process words one at a time

C Focus on important parts of the input sequence

D Reduce the size of the input

Self-attention helps the model focus on relevant words in the sequence to understand context better.

Why is positional encoding necessary in Transformers?

A To increase vocabulary size

B To speed up training

C To reduce model size

D Because Transformers do not process data sequentially

Positional encoding tells the model the order of words since Transformers look at all words at once.

Which part of the Transformer generates the final output?

A Decoder

B Encoder

C Self-attention layer

D Positional encoding

The Decoder uses the Encoder's information to produce the output sequence.

How does the Transformer improve over RNNs?

A Processes sequences in parallel using self-attention

B Processes sequences strictly one word at a time

C Uses fewer layers

D Ignores context

Transformers process all words simultaneously, making training faster and capturing long-range dependencies better.

Explain how self-attention works in the Transformer architecture and why it is important.

Think about how the model decides which words to focus on when reading a sentence.

Describe the roles of the Encoder and Decoder in the Transformer model.

Consider the flow from input to output in a translation task.