Aspect-based sentiment analysis in NLP - ML Experiment: Train & Evaluate

Experiment - Aspect-based sentiment analysis

Problem:You want to analyze customer reviews to find the sentiment (positive, negative, neutral) about specific aspects like 'battery', 'screen', or 'price' in product reviews.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Validation loss: 0.85

Issue:The model overfits: it performs very well on training data but poorly on validation data, showing it does not generalize well.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You can only modify the model architecture and training hyperparameters.

You cannot change the dataset or add more data.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

NLP

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Assume X_train, y_train, X_val, y_val are preprocessed and ready
vocab_size = 10000
embedding_dim = 100
max_length = 100

model = Sequential([
    Embedding(vocab_size, embedding_dim, input_length=max_length),
    LSTM(64, return_sequences=False),
    Dropout(0.5),
    Dense(32, activation='relu'),
    Dropout(0.5),
    Dense(3, activation='softmax')  # 3 classes: positive, negative, neutral
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)

history = model.fit(X_train, y_train,
                    epochs=20,
                    batch_size=64,
                    validation_data=(X_val, y_val),
                    callbacks=[early_stop])

Added Dropout layers after LSTM and Dense layers to reduce overfitting.

Reduced LSTM units from 128 to 64 to simplify the model.

Added EarlyStopping callback to stop training early when validation loss stops improving.

Lowered learning rate to 0.001 for smoother training.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Validation loss 0.85

After: Training accuracy 90%, Validation accuracy 87%, Validation loss 0.45

Adding dropout and early stopping helped reduce overfitting. The model now generalizes better to new data, improving validation accuracy while slightly lowering training accuracy.

Bonus Experiment

Try using a pretrained language model like BERT for aspect-based sentiment analysis to improve accuracy further.

💡 Hint

Use a pretrained BERT model with a classification head and fine-tune it on your dataset.