BERT fine-tuning for classification in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Model Choice

intermediate

Choosing the correct BERT model for fine-tuning

You want to fine-tune a BERT model for a text classification task with 3 classes. Which pretrained model is the best starting point?

A bert-base-uncased

B gpt-2

C bert-large-cased

D resnet50

💡 Hint

Pick a model designed for language understanding and classification.

bert-base-uncased is a common pretrained BERT model suitable for classification tasks. GPT-2 is a language generation model, and ResNet50 is for images.

❓ Predict Output

intermediate

Output shape after BERT forward pass

Given this code snippet, what is the shape of outputs.logits?

NLP

from transformers import BertForSequenceClassification, BertTokenizer
import torch

model = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=4)
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
inputs = tokenizer(['Hello world', 'Test sentence'], padding=True, return_tensors='pt')
outputs = model(**inputs)
logits_shape = outputs.logits.shape

A (768, 4)

B (4, 2)

C (2, 768)

D (2, 4)

💡 Hint

Check batch size and number of labels.

The batch size is 2 (two sentences), and the model outputs logits for 4 classes, so shape is (2, 4).

❓ Hyperparameter

advanced

Choosing learning rate for BERT fine-tuning

You are fine-tuning BERT for classification. Which learning rate is most appropriate to start with?

A 5e-5

B 0.1

C 1.0

D 0.000001

💡 Hint

Typical fine-tuning learning rates are small but not too tiny.

5e-5 is a common learning rate for fine-tuning BERT. Too large (0.1, 1.0) causes training instability. Too small (1e-6) slows training.

❓ Metrics

advanced

Evaluating BERT classification performance

After fine-tuning BERT on a 3-class classification task, you get these predictions and true labels:

preds = [0, 2, 1, 1, 0]
labels = [0, 1, 1, 2, 0]

What is the accuracy?

A 0.8

B 0.4

C 0.6

D 0.2

💡 Hint

Count how many predictions match labels exactly.

Matches at indices 0, 2, and 4 → 3 correct out of 5 → accuracy 0.6.

🔧 Debug

expert

Identifying error in BERT fine-tuning code

What error does this code raise?

from transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)
inputs = {'input_ids': [[101, 2054, 2003, 1996, 2562, 102]], 'attention_mask': [[1, 1, 1, 1, 1, 1]]}
outputs = model(**inputs)
loss = outputs.loss

A AttributeError: 'SequenceClassifierOutput' object has no attribute 'loss'

B RuntimeError: Expected tensor for input_ids, got list

C TypeError: forward() missing required positional argument 'labels'

D No error, code runs successfully

💡 Hint

Check the type/format of input_ids and attention_mask.

The model expects PyTorch tensors for input_ids and attention_mask, but lists are provided instead, causing a RuntimeError when trying to process them in the embedding layer.