Sentiment analysis in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Sentiment analysis

Which metric matters for Sentiment analysis and WHY

In sentiment analysis, we often want to know how well the model can correctly identify positive and negative feelings in text. The key metrics are Precision, Recall, and F1-score. Precision tells us how many of the texts labeled as positive (or negative) are actually correct. Recall tells us how many of the actual positive (or negative) texts the model found. F1-score balances both. These metrics matter because sometimes missing a negative review (low recall) or wrongly labeling a positive review as negative (low precision) can affect decisions.

Confusion matrix example

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    80    |   20    
      Negative           |    15    |   85    

      Total samples = 80 + 20 + 15 + 85 = 200

      Precision (Positive) = TP / (TP + FP) = 80 / (80 + 15) = 0.842
      Recall (Positive) = TP / (TP + FN) = 80 / (80 + 20) = 0.8
      F1-score (Positive) = 2 * (0.842 * 0.8) / (0.842 + 0.8) ≈ 0.82
    

Precision vs Recall tradeoff with examples

Imagine a company uses sentiment analysis to detect unhappy customers from reviews.

• High Precision: If the model says a review is negative, it is very likely correct. This avoids bothering happy customers by mistake.
• High Recall: The model finds most unhappy customers, even if some happy ones are wrongly flagged. This helps catch all problems but may annoy some happy customers.

Choosing which to prioritize depends on the goal: if avoiding false alarms is key, prioritize precision; if catching all unhappy customers is key, prioritize recall.

What "good" vs "bad" metric values look like for Sentiment analysis

• Good: Precision and recall above 0.8, F1-score around 0.8 or higher. This means the model correctly identifies most sentiments with few mistakes.
• Bad: Precision or recall below 0.5 means many wrong labels or many missed sentiments. For example, precision 0.4 means more than half predicted positives are wrong.
• Accuracy: Can be misleading if data is unbalanced (e.g., 90% positive reviews). High accuracy but low recall on negative class is bad.

Common pitfalls in Sentiment analysis metrics

• Accuracy paradox: High accuracy can hide poor performance on minority classes.
• Data leakage: If test data leaks into training, metrics look unrealistically high.
• Overfitting: Very high training metrics but low test metrics mean the model memorizes training data, not generalizing well.
• Ignoring class imbalance: Not using metrics like F1-score or balanced accuracy can mislead evaluation.

Self-check question

Your sentiment analysis model has 98% accuracy but only 12% recall on negative reviews. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall on negative reviews means the model misses most unhappy customers. Even though accuracy is high, it likely predicts most reviews as positive. This hurts the goal of detecting negative sentiment.

Key Result

Precision, recall, and F1-score are key to evaluate sentiment analysis models because they balance correct detection and missed cases.

Practice

1. What is the main goal of sentiment analysis in TensorFlow?

easy

A. To translate text into another language

B. To classify text as positive or negative

C. To generate new text automatically

D. To count the number of words in a sentence

Solution

1. Step 1: Understand sentiment analysis purpose

   Sentiment analysis is used to find the feeling or opinion in text, usually positive or negative.

2. Step 2: Match goal with options

   Among the options, only classifying text as positive or negative matches sentiment analysis.

3. Final Answer:

   To classify text as positive or negative -> Option B

4. Quick Check:

   Sentiment analysis = classify positive/negative [OK]

Hint: Sentiment analysis = positive or negative text classification [OK]

Common Mistakes:

• Confusing sentiment analysis with translation
• Thinking it generates new text
• Believing it only counts words

2. Which TensorFlow layer is commonly used to convert words into numbers for sentiment analysis?

easy

A. Dense layer

B. Flatten layer

C. Dropout layer

D. Embedding layer

Solution

1. Step 1: Identify layer for word representation

   Embedding layers convert words into numeric vectors that models can understand.

2. Step 2: Compare with other layers

   Dense layers are for fully connected neurons, Dropout prevents overfitting, Flatten reshapes data, none convert words to numbers.

3. Final Answer:

   Embedding layer -> Option D

4. Quick Check:

   Embedding layer = word to number conversion [OK]

Hint: Embedding layer turns words into numbers for models [OK]

Common Mistakes:

• Choosing Dense layer for word conversion
• Confusing Dropout with embedding
• Using Flatten layer incorrectly

3. What will be the output shape of the embedding layer if input shape is (None, 10) and embedding dimension is 16?

medium

A. (None, 10, 16)

B. (None, 16, 10)

C. (10, 16)

D. (None, 10)

Solution

1. Step 1: Understand embedding input and output shapes

   Input shape (None, 10) means batch size unknown, sequence length 10. Embedding outputs a vector per word with size 16.

2. Step 2: Calculate output shape

   Output shape is (batch_size, sequence_length, embedding_dim) = (None, 10, 16).

3. Final Answer:

   (None, 10, 16) -> Option A

4. Quick Check:

   Embedding output shape = (batch, seq_len, embed_dim) [OK]

Hint: Embedding output shape = input length x embedding size [OK]

Common Mistakes:

• Swapping sequence length and embedding dimension
• Ignoring batch size dimension
• Confusing input and output shapes

4. Identify the error in this TensorFlow model snippet for sentiment analysis:

model = tf.keras.Sequential([
  tf.keras.layers.Embedding(input_dim=1000, output_dim=16, input_length=10),
  tf.keras.layers.Dense(1, activation='sigmoid')
])

medium

A. Dense layer activation should be 'relu' for binary output

B. Embedding layer input_dim should be output_dim

C. Missing Flatten or GlobalPooling layer before Dense

D. input_length parameter is not allowed in Embedding layer

Solution

1. Step 1: Analyze model layer sequence

   Embedding outputs 3D tensor (batch, seq_len, embed_dim). Dense expects 2D input, so flattening or pooling is needed.

2. Step 2: Identify missing layer

   Without Flatten or GlobalPooling, Dense layer will raise shape error.

3. Final Answer:

   Missing Flatten or GlobalPooling layer before Dense -> Option C

4. Quick Check:

   Embedding output must be flattened before Dense [OK]

Hint: Add Flatten or pooling before Dense after Embedding [OK]

Common Mistakes:

• Confusing input_dim and output_dim in Embedding
• Using wrong activation for binary classification
• Thinking input_length is invalid parameter

5. You want to improve a sentiment analysis model's accuracy on short reviews. Which approach is best?

hard

A. Increase embedding dimension and add a Bidirectional LSTM layer

B. Reduce embedding dimension and remove all recurrent layers

C. Use only a Dense layer without embedding

D. Train on fewer examples to avoid overfitting

Solution

1. Step 1: Consider model improvements for short text

   Increasing embedding dimension helps capture more word features; Bidirectional LSTM captures context from both directions.

2. Step 2: Evaluate other options

   Reducing embedding or removing recurrent layers loses information; training on fewer examples reduces learning quality.

3. Final Answer:

   Increase embedding dimension and add a Bidirectional LSTM layer -> Option A

4. Quick Check:

   More embedding + Bidirectional LSTM = better short text accuracy [OK]

Hint: Use bigger embeddings and bidirectional LSTM for short texts [OK]

Common Mistakes:

• Thinking smaller embeddings improve accuracy
• Removing recurrent layers harms context understanding
• Training on fewer examples reduces model learning