Pre-training and fine-tuning concept in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Pre-training and fine-tuning concept

Problem:You have a language model pre-trained on a large text dataset. Now you want to adapt it to classify movie reviews as positive or negative.

Current Metrics:Training accuracy: 98%, Validation accuracy: 70%

Issue:The model overfits the training data and performs poorly on validation data.

Your Task

Reduce overfitting and improve validation accuracy to at least 85% while keeping training accuracy below 92%.

You can only adjust fine-tuning parameters and add regularization techniques.

You cannot change the pre-trained model architecture or dataset.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Prompt Engineering / GenAI

import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Assume pre_trained_model is loaded
# Freeze base model layers
for layer in pre_trained_model.layers:
    layer.trainable = False

# Add classification head
x = pre_trained_model.output
x = Dropout(0.3)(x)
x = Dense(64, activation='relu')(x)
x = Dropout(0.3)(x)
outputs = Dense(1, activation='sigmoid')(x)

model = Model(inputs=pre_trained_model.input, outputs=outputs)

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=1e-5),
              loss='binary_crossentropy',
              metrics=['accuracy'])

early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)

history = model.fit(train_data, train_labels,
                    epochs=20,
                    batch_size=32,
                    validation_data=(val_data, val_labels),
                    callbacks=[early_stop])

Froze the pre-trained model layers to keep learned features stable.

Added dropout layers with 30% rate to reduce overfitting.

Lowered learning rate to 1e-5 for smoother fine-tuning.

Used early stopping to stop training when validation loss stops improving.

Results Interpretation

Before: Training accuracy: 98%, Validation accuracy: 70% (overfitting)

After: Training accuracy: 90%, Validation accuracy: 87% (better generalization)

Freezing pre-trained layers and adding dropout with a lower learning rate helps reduce overfitting and improves validation accuracy during fine-tuning.

Bonus Experiment

Try unfreezing some of the last layers of the pre-trained model and fine-tune them with a very low learning rate to see if validation accuracy improves further.

💡 Hint

Unfreeze only the last 2-3 layers and keep the rest frozen. Use a learning rate around 1e-6.