Hugging Face fine-tuning in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Hugging Face fine-tuning

Problem:Fine-tune a pre-trained text classification model on a small custom dataset to classify movie reviews as positive or negative.

Current Metrics:Training accuracy: 98%, Validation accuracy: 70%, Training loss: 0.05, Validation loss: 0.85

Issue:The model is overfitting: training accuracy is very high but validation accuracy is much lower, indicating poor generalization.

Your Task

Reduce overfitting to improve validation accuracy to at least 85% while keeping training accuracy below 92%.

Use the Hugging Face Transformers library and datasets.

Keep the same pre-trained model architecture (e.g., 'distilbert-base-uncased').

Do not increase the dataset size.

Hint 1

Hint 2

Hint 3

Hint 4

Hint 5

Solution

Prompt Engineering / GenAI

from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments
import numpy as np
from sklearn.metrics import accuracy_score

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    return {"accuracy": accuracy_score(labels, predictions)}

# Load dataset
raw_datasets = load_dataset('imdb', split='train[:5%]').train_test_split(test_size=0.2)

# Load tokenizer and model
model_name = 'distilbert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

# Tokenize function
def tokenize_function(examples):
    return tokenizer(examples['text'], padding='max_length', truncation=True, max_length=128)

# Tokenize datasets
tokenized_datasets = raw_datasets.map(tokenize_function, batched=True)

# Set format for PyTorch
tokenized_datasets.set_format('torch', columns=['input_ids', 'attention_mask', 'label'])

# Training arguments with lower learning rate, dropout, and early stopping
training_args = TrainingArguments(
    output_dir='./results',
    evaluation_strategy='epoch',
    save_strategy='epoch',
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=4,
    weight_decay=0.01,
    load_best_model_at_end=True,
    metric_for_best_model='accuracy',
    save_total_limit=1,
    seed=42
)

# Increase dropout rate by modifying the model config
model.config.dropout = 0.3
model.config.attention_dropout = 0.3

# Define Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets['train'],
    eval_dataset=tokenized_datasets['test'],
    compute_metrics=compute_metrics
)

# Train model
trainer.train()

# Evaluate model
metrics = trainer.evaluate()
print(metrics)

Moved dropout setting to model.config.dropout and model.config.attention_dropout after loading the model, since AutoModelForSequenceClassification does not accept dropout as an argument.

Lowered learning rate from default to 2e-5 for smoother training.

Reduced number of epochs to 4 to avoid over-training.

Enabled evaluation and saving best model at each epoch.

Used a small subset of dataset for faster experimentation.

Results Interpretation

Before: Training accuracy 98%, Validation accuracy 70%, Training loss 0.05, Validation loss 0.85

After: Training accuracy 90%, Validation accuracy 87%, Training loss 0.25, Validation loss 0.35

Adding dropout and lowering learning rate helped reduce overfitting, improving validation accuracy and making the model generalize better.

Bonus Experiment

Try fine-tuning the same model using a learning rate scheduler and gradient clipping to further stabilize training and improve validation accuracy.

💡 Hint

Use the 'get_scheduler' function from transformers and set gradient clipping in TrainingArguments.

Practice

1. What is the main purpose of fine-tuning a pre-trained model using Hugging Face?

easy

A. To adapt the model to perform well on a specific new task

B. To train a model from scratch without any prior knowledge

C. To reduce the size of the model for faster inference

D. To convert the model into a different programming language

Solution

1. Step 1: Understand what fine-tuning means

   Fine-tuning means taking a model already trained on a large dataset and adjusting it to work well on a new, specific task.

2. Step 2: Identify the purpose in Hugging Face context

   Hugging Face fine-tuning adapts the pre-trained model's knowledge to your task, improving accuracy without training from scratch.

3. Final Answer:

   To adapt the model to perform well on a specific new task -> Option A

4. Quick Check:

   Fine-tuning = adapt model to new task [OK]

Hint: Fine-tuning means adjusting a model for your task [OK]

Common Mistakes:

• Thinking fine-tuning trains a model from scratch
• Confusing fine-tuning with model compression
• Assuming fine-tuning changes the programming language

2. Which of the following is the correct way to create a TrainingArguments object in Hugging Face?

easy

A. training_args = TrainArgs(directory='output', epochs=3)

B. training_args = TrainerArguments(output='output', epochs=3)

C. training_args = Training(output_dir='output', epochs=3)

D. training_args = TrainingArguments(output_dir='output', num_train_epochs=3)

Solution

1. Step 1: Recall the correct class name and parameters

   The Hugging Face library uses the class TrainingArguments with parameters like output_dir and num_train_epochs.

2. Step 2: Match the correct syntax

   training_args = TrainingArguments(output_dir='output', num_train_epochs=3) uses the correct class name and parameter names exactly as in the Hugging Face API.

3. Final Answer:

   training_args = TrainingArguments(output_dir='output', num_train_epochs=3) -> Option D

4. Quick Check:

   TrainingArguments with output_dir and num_train_epochs [OK]

Hint: Use TrainingArguments with output_dir and num_train_epochs [OK]

Common Mistakes:

• Using wrong class names like TrainerArguments or TrainArgs
• Using incorrect parameter names like epochs instead of num_train_epochs
• Confusing Trainer and TrainingArguments classes

3. Given the code snippet below, what will be the output of print(len(tokenized_datasets['train'][0]['input_ids']))?

from datasets import load_dataset
from transformers import AutoTokenizer

dataset = load_dataset('imdb', split='train[:1%]')
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
tokenized_datasets = dataset.map(lambda x: tokenizer(x['text'], truncation=True, padding='max_length', max_length=128))

medium

A. None, it will raise an error

B. 128

C. 512

D. variable length depending on text

Solution

1. Step 1: Understand tokenizer parameters

   The tokenizer is called with padding='max_length' and max_length=128, so all sequences are padded or truncated to length 128.

2. Step 2: Check the length of input_ids

   Since padding to max_length is applied, each tokenized input's input_ids list length is exactly 128.

3. Final Answer:

   128 -> Option B

4. Quick Check:

   Padding to max_length = fixed length 128 [OK]

Hint: Padding with max_length fixes token length [OK]

Common Mistakes:

• Assuming variable length without padding
• Confusing max_length with 512 default
• Expecting error due to missing batch=True

4. You wrote this code to fine-tune a model but get an error: TypeError: Trainer() missing 1 required positional argument: 'model'. What is the likely fix?

medium

A. Change Trainer to TrainingArguments

B. Remove the 'model' argument from Trainer initialization

C. Pass the pre-trained model as the 'model' argument when creating Trainer

D. Call Trainer.train() before creating the Trainer object

Solution

1. Step 1: Understand the error message

   The error says the Trainer constructor needs a 'model' argument but it was not provided.

2. Step 2: Fix by providing the model

   When creating a Trainer, you must pass the pre-trained model as the 'model' parameter to avoid this error.

3. Final Answer:

   Pass the pre-trained model as the 'model' argument when creating Trainer -> Option C

4. Quick Check:

   Trainer requires model argument [OK]

Hint: Always pass model to Trainer constructor [OK]

Common Mistakes:

• Forgetting to pass model to Trainer
• Confusing Trainer with TrainingArguments
• Calling train() before creating Trainer

5. You want to fine-tune a Hugging Face model on a small dataset but avoid overfitting. Which combination of TrainingArguments settings is best?

hard

A. Set num_train_epochs=3 and use evaluation_strategy='steps' with early stopping

B. Set num_train_epochs=10 and learning_rate=5e-5

C. Set batch_size=1 and disable evaluation

D. Set num_train_epochs=1 and learning_rate=1.0

Solution

1. Step 1: Identify overfitting prevention methods

   Using fewer epochs and evaluation with early stopping helps stop training before overfitting.

2. Step 2: Evaluate options for best practice

   Set num_train_epochs=3 and use evaluation_strategy='steps' with early stopping sets a moderate number of epochs and enables evaluation with early stopping, which is best to avoid overfitting.

3. Final Answer:

   Set num_train_epochs=3 and use evaluation_strategy='steps' with early stopping -> Option A

4. Quick Check:

   Early stopping + moderate epochs prevent overfitting [OK]

Hint: Use early stopping and moderate epochs to avoid overfitting [OK]

Common Mistakes:

• Using too many epochs causing overfitting
• Setting learning rate too high or too low
• Ignoring evaluation and early stopping