Why different transformers serve different tasks in NLP - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why do transformers have different architectures for different tasks?

Transformers are used in many tasks like translation, text classification, and question answering. Why do we need different transformer models for these tasks?

A Because transformers only work for translation tasks and cannot be used for other tasks without changing the language.

B Because transformers use different programming languages for different tasks.

C Because transformers are slow and need to be replaced by different models for each task.

D Because each task requires a different way to process input and output, so transformer models are designed with specific layers or heads to handle those needs.

💡 Hint

Think about how the output of a model changes depending on the task.

Different tasks require different outputs and ways to understand input. For example, translation needs to generate sentences, classification needs to assign labels. So, transformer models are adapted with specific layers or heads to fit the task.

❓ Model Choice

intermediate

Choosing the right transformer for sentiment analysis

You want to build a model to classify movie reviews as positive or negative. Which transformer model is best suited for this task?

A GPT-3 used as a language generator without fine-tuning

B A convolutional neural network without transformers

C BERT with a classification head on top

D Transformer model designed only for machine translation

💡 Hint

Think about which model is designed to understand sentence meaning and output labels.

BERT is designed to understand sentence meaning and can be fine-tuned with a classification head to output sentiment labels. GPT-3 without fine-tuning is a generator, and translation models focus on language conversion, not classification.

❓ Predict Output

advanced

Output shape of transformer model for question answering

Consider a transformer model fine-tuned for question answering. The input is a batch of 2 sequences, each with 10 tokens. The model outputs start and end logits for answer spans. What is the shape of the output logits?

NLP

import torch
batch_size = 2
seq_len = 10
start_logits = torch.randn(batch_size, seq_len)
end_logits = torch.randn(batch_size, seq_len)
print(start_logits.shape, end_logits.shape)

A (2, 10) (2, 10)

B (10, 2) (10, 2)

C (2, 1) (2, 1)

D (20,) (20,)

💡 Hint

Think about how logits correspond to tokens in each sequence for each batch item.

For question answering, the model outputs two sets of logits per token position: one for start and one for end. Since batch size is 2 and sequence length is 10, output shapes are (2, 10) for both start and end logits.

❓ Hyperparameter

advanced

Effect of number of attention heads in transformer models

What is the main effect of increasing the number of attention heads in a transformer model?

A It allows the model to focus on different parts of the input simultaneously, improving its ability to capture diverse relationships.

B It reduces the model size and speeds up training by using fewer parameters.

C It disables the self-attention mechanism and uses only feed-forward layers.

D It changes the output format from text to images.

💡 Hint

Think about what multiple attention heads do in the transformer.

Multiple attention heads let the model look at different parts of the input at once, capturing various relationships and improving understanding. It does not reduce size or change output type.

🔧 Debug

expert

Why does this transformer model output all zeros?

You fine-tuned a transformer for text classification, but the model always outputs zeros for predictions. What is the most likely cause?

import torch
from transformers import BertForSequenceClassification, BertTokenizer

model = BertForSequenceClassification.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

inputs = tokenizer('Hello world', return_tensors='pt')
outputs = model(**inputs)
print(outputs.logits)

A The tokenizer is incorrect and returns empty input tensors.

B The model is in evaluation mode but was never trained, so outputs are near zero logits.

C The model architecture is wrong and does not produce logits.

D The input text is too short, so the model outputs zeros.

💡 Hint

Think about what happens if you use a pretrained model without fine-tuning for classification.

The pretrained BERT model for sequence classification is randomly initialized for the classification head and not trained yet. So logits are near zero or random but often close to zero. Tokenizer and architecture are correct, and input length does not cause zero outputs.