Aspect-based sentiment analysis in NLP - ML Experiment: Train & Evaluate

Experiment - Aspect-based sentiment analysis

Problem:You want to analyze customer reviews to find the sentiment (positive, negative, neutral) about specific aspects like 'battery', 'screen', or 'price' in product reviews.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Validation loss: 0.85

Issue:The model overfits: it performs very well on training data but poorly on validation data, showing it does not generalize well.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You can only modify the model architecture and training hyperparameters.

You cannot change the dataset or add more data.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

NLP

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Assume X_train, y_train, X_val, y_val are preprocessed and ready
vocab_size = 10000
embedding_dim = 100
max_length = 100

model = Sequential([
    Embedding(vocab_size, embedding_dim, input_length=max_length),
    LSTM(64, return_sequences=False),
    Dropout(0.5),
    Dense(32, activation='relu'),
    Dropout(0.5),
    Dense(3, activation='softmax')  # 3 classes: positive, negative, neutral
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)

history = model.fit(X_train, y_train,
                    epochs=20,
                    batch_size=64,
                    validation_data=(X_val, y_val),
                    callbacks=[early_stop])

Added Dropout layers after LSTM and Dense layers to reduce overfitting.

Reduced LSTM units from 128 to 64 to simplify the model.

Added EarlyStopping callback to stop training early when validation loss stops improving.

Lowered learning rate to 0.001 for smoother training.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Validation loss 0.85

After: Training accuracy 90%, Validation accuracy 87%, Validation loss 0.45

Adding dropout and early stopping helped reduce overfitting. The model now generalizes better to new data, improving validation accuracy while slightly lowering training accuracy.

Bonus Experiment

Try using a pretrained language model like BERT for aspect-based sentiment analysis to improve accuracy further.

💡 Hint

Use a pretrained BERT model with a classification head and fine-tune it on your dataset.

Practice

1. What is the main goal of aspect-based sentiment analysis?

easy

A. To translate text from one language to another

B. To count the number of words in a sentence

C. To find feelings about specific parts or features in text

D. To generate new text based on input

Solution

1. Step 1: Understand the concept of aspect-based sentiment analysis

   It focuses on identifying opinions about specific parts or features, not the whole text.

2. Step 2: Compare options with this concept

   Only To find feelings about specific parts or features in text matches this goal; others describe unrelated tasks.

3. Final Answer:

   To find feelings about specific parts or features in text -> Option C

4. Quick Check:

   Aspect-based sentiment = specific parts sentiment [OK]

Hint: Focus on 'specific parts' in the question to find the goal [OK]

Common Mistakes:

• Confusing overall sentiment with aspect-level sentiment
• Thinking it translates or generates text
• Mixing sentiment analysis with word counting

2. Which Python library is commonly used for simple aspect-based sentiment analysis?

easy

A. Matplotlib

B. NumPy

C. Flask

D. TextBlob

Solution

1. Step 1: Identify libraries related to text sentiment

   TextBlob is a simple library for sentiment analysis and text processing.

2. Step 2: Eliminate unrelated libraries

   NumPy is for numbers, Matplotlib for plotting, Flask for web apps, so they don't fit.

3. Final Answer:

   TextBlob -> Option D

4. Quick Check:

   TextBlob = simple sentiment tool [OK]

Hint: Pick the library known for text sentiment, not numbers or web [OK]

Common Mistakes:

• Choosing NumPy or Matplotlib which are not for sentiment
• Confusing Flask as a sentiment tool
• Not knowing TextBlob's purpose

3. Given this Python code snippet for aspect sentiment, what is the output?

from textblob import TextBlob
text = "The battery life is great but the screen is dull."
aspects = ['battery life', 'screen']
results = {}
for aspect in aspects:
    blob = TextBlob(text)
    if aspect in text:
        sentiment = blob.sentiment.polarity
        results[aspect] = 'positive' if sentiment > 0 else 'negative'
print(results)

medium

A. {'battery life': 'positive', 'screen': 'positive'}

B. {'battery life': 'positive', 'screen': 'negative'}

C. {'battery life': 'negative', 'screen': 'negative'}

D. SyntaxError

Solution

1. Step 1: Understand the sentiment polarity calculation

   TextBlob calculates overall sentiment polarity of the whole text, which is positive because 'battery life is great' outweighs 'screen is dull'.

2. Step 2: Check how results are assigned

   For each aspect found in text, sentiment polarity is checked once (overall), so both aspects get 'positive'.

3. Final Answer:

   {'battery life': 'positive', 'screen': 'positive'} -> Option A

4. Quick Check:

   Overall sentiment positive -> both aspects positive [OK]

Hint: TextBlob sentiment is overall, so all aspects get same polarity [OK]

Common Mistakes:

• Assuming aspect-specific sentiment without extra processing
• Expecting different sentiment for each aspect
• Thinking code has syntax errors

4. Identify the error in this aspect-based sentiment analysis code snippet:

from textblob import TextBlob
text = "The food was tasty but the service was slow."
aspects = ['food', 'service']
results = {}
for aspect in aspects:
    blob = TextBlob(aspect)
    sentiment = blob.sentiment.polarity
    results[aspect] = 'positive' if sentiment > 0 else 'negative'
print(results)

medium

A. The aspects list is empty

B. TextBlob is applied to aspect text, not the full sentence

C. The results dictionary is not initialized

D. The print statement is missing parentheses

Solution

1. Step 1: Check what text is analyzed by TextBlob

   TextBlob is called on the aspect word only, not the full sentence, so sentiment is meaningless.

2. Step 2: Identify correct usage

   TextBlob should analyze the full sentence or relevant sentence part, not just the aspect word.

3. Final Answer:

   TextBlob is applied to aspect text, not the full sentence -> Option B

4. Quick Check:

   TextBlob needs full text, not just aspect [OK]

Hint: Check what text TextBlob analyzes--aspect or full sentence? [OK]

Common Mistakes:

• Thinking aspects list is empty
• Ignoring that results dict is initialized
• Assuming print syntax error in Python 3

5. You want to improve aspect-based sentiment analysis by focusing on sentences mentioning each aspect separately. Which approach is best?

hard

A. Split text into sentences, analyze sentiment only for sentences containing the aspect

B. Analyze the entire text sentiment once and assign it to all aspects

C. Ignore sentences and analyze only aspect words with TextBlob

D. Use random sentiment values for each aspect

Solution

1. Step 1: Understand the problem with overall sentiment

   Overall sentiment mixes all opinions, so it can't tell which aspect is positive or negative.

2. Step 2: Use sentence-level analysis for precision

   Splitting text into sentences and analyzing only those mentioning the aspect gives accurate aspect sentiment.

3. Final Answer:

   Split text into sentences, analyze sentiment only for sentences containing the aspect -> Option A

4. Quick Check:

   Sentence-level sentiment = better aspect accuracy [OK]

Hint: Analyze sentences mentioning aspect, not whole text [OK]

Common Mistakes:

• Using overall sentiment for all aspects
• Analyzing only aspect words without context
• Assigning random sentiment values