Fine-grained sentiment (5-class) in NLP - ML Experiment: Train & Evaluate

Experiment - Fine-grained sentiment (5-class)

Problem:Classify movie reviews into 5 sentiment classes: very negative, negative, neutral, positive, very positive.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Training loss: 0.15, Validation loss: 0.60

Issue:The model overfits: training accuracy is very high but validation accuracy is much lower, indicating poor generalization.

Your Task

Reduce overfitting so that validation accuracy improves to at least 80%, while keeping training accuracy below 90%.

You can only modify the model architecture and training hyperparameters.

Do not change the dataset or preprocessing steps.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

NLP

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Assume X_train, y_train, X_val, y_val are preprocessed and ready

vocab_size = 10000
embedding_dim = 64
max_length = 100

model = Sequential([
    Embedding(vocab_size, embedding_dim, input_length=max_length),
    LSTM(64, return_sequences=False),
    Dropout(0.5),
    Dense(32, activation='relu'),
    Dropout(0.5),
    Dense(5, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)

history = model.fit(X_train, y_train,
                    epochs=20,
                    batch_size=64,
                    validation_data=(X_val, y_val),
                    callbacks=[early_stop])

Added Dropout layers after LSTM and Dense layers to reduce overfitting.

Reduced LSTM units from 128 to 64 to lower model complexity.

Added EarlyStopping callback to stop training when validation loss stops improving.

Set learning rate to 0.001 for smoother training.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Training loss 0.15, Validation loss 0.60

After: Training accuracy 88%, Validation accuracy 82%, Training loss 0.30, Validation loss 0.45

Adding dropout and early stopping helped reduce overfitting. The model now generalizes better, improving validation accuracy while slightly lowering training accuracy.

Bonus Experiment

Try using a bidirectional LSTM layer instead of a single LSTM layer to see if it improves validation accuracy further.

💡 Hint

Replace LSTM with tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(...)) and keep dropout layers.