Sentiment analysis in TensorFlow - Model Pipeline Trace

Model Pipeline - Sentiment analysis

This pipeline reads text reviews, cleans and converts them into numbers, trains a model to learn positive or negative feelings, and then predicts the sentiment of new reviews.

Data Flow - 5 Stages

1Raw data input

1000 rows x 1 column→Load text reviews and labels (positive/negative)→1000 rows x 1 column

["I love this movie!", "This was a bad experience."]

↓

2Text preprocessing

1000 rows x 1 column→Lowercase, remove punctuation, tokenize words→1000 rows x variable length sequences

[["i", "love", "this", "movie"], ["this", "was", "a", "bad", "experience"]]

↓

3Text to sequence

1000 rows x variable length sequences→Convert words to integer indexes using vocabulary→1000 rows x 10 columns

[[12, 45, 7, 89, 0, 0, 0, 0, 0, 0], [7, 23, 4, 56, 78, 0, 0, 0, 0, 0]]

↓

4Train/test split

1000 rows x 10 columns→Split data into training (80%) and testing (20%) sets→Train: 800 rows x 10 columns, Test: 200 rows x 10 columns

Train sample: [12, 45, 7, 89, 0, 0, 0, 0, 0, 0]

↓

5Model training

Train: 800 rows x 10 columns→Train embedding + LSTM + dense layers to classify sentiment→Trained model

Model learns to map sequences to sentiment labels

Training Trace - Epoch by Epoch

Epoch 1: 0.65 #######
Epoch 2: 0.50 #####
Epoch 3: 0.40 ####
Epoch 4: 0.35 ###
Epoch 5: 0.30 ##

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.60	Model starts learning, accuracy above random
2	0.50	0.75	Loss decreases, accuracy improves significantly
3	0.40	0.82	Model continues to improve
4	0.35	0.86	Training converging well
5	0.30	0.89	Good accuracy, loss low

Prediction Trace - 5 Layers

Layer 1: Input text

Layer 2: Text to sequence

Layer 3: Embedding layer

Layer 4: LSTM layer

Layer 5: Dense + Sigmoid output

Model Quiz - 3 Questions

What happens to the text data during the 'Text to sequence' stage?

A Numbers are converted back to words

B Words are converted to numbers representing their position in vocabulary

C Text is split into sentences

D Text is translated to another language

Key Insight

This visualization shows how raw text is transformed into numbers, then learned by a model to predict sentiment. Watching loss decrease and accuracy increase confirms the model is learning well.

Practice

1. What is the main goal of sentiment analysis in TensorFlow?

easy

A. To translate text into another language

B. To classify text as positive or negative

C. To generate new text automatically

D. To count the number of words in a sentence

Solution

1. Step 1: Understand sentiment analysis purpose

   Sentiment analysis is used to find the feeling or opinion in text, usually positive or negative.

2. Step 2: Match goal with options

   Among the options, only classifying text as positive or negative matches sentiment analysis.

3. Final Answer:

   To classify text as positive or negative -> Option B

4. Quick Check:

   Sentiment analysis = classify positive/negative [OK]

Hint: Sentiment analysis = positive or negative text classification [OK]

Common Mistakes:

• Confusing sentiment analysis with translation
• Thinking it generates new text
• Believing it only counts words

2. Which TensorFlow layer is commonly used to convert words into numbers for sentiment analysis?

easy

A. Dense layer

B. Flatten layer

C. Dropout layer

D. Embedding layer

Solution

1. Step 1: Identify layer for word representation

   Embedding layers convert words into numeric vectors that models can understand.

2. Step 2: Compare with other layers

   Dense layers are for fully connected neurons, Dropout prevents overfitting, Flatten reshapes data, none convert words to numbers.

3. Final Answer:

   Embedding layer -> Option D

4. Quick Check:

   Embedding layer = word to number conversion [OK]

Hint: Embedding layer turns words into numbers for models [OK]

Common Mistakes:

• Choosing Dense layer for word conversion
• Confusing Dropout with embedding
• Using Flatten layer incorrectly

3. What will be the output shape of the embedding layer if input shape is (None, 10) and embedding dimension is 16?

medium

A. (None, 10, 16)

B. (None, 16, 10)

C. (10, 16)

D. (None, 10)

Solution

1. Step 1: Understand embedding input and output shapes

   Input shape (None, 10) means batch size unknown, sequence length 10. Embedding outputs a vector per word with size 16.

2. Step 2: Calculate output shape

   Output shape is (batch_size, sequence_length, embedding_dim) = (None, 10, 16).

3. Final Answer:

   (None, 10, 16) -> Option A

4. Quick Check:

   Embedding output shape = (batch, seq_len, embed_dim) [OK]

Hint: Embedding output shape = input length x embedding size [OK]

Common Mistakes:

• Swapping sequence length and embedding dimension
• Ignoring batch size dimension
• Confusing input and output shapes

4. Identify the error in this TensorFlow model snippet for sentiment analysis:

model = tf.keras.Sequential([
  tf.keras.layers.Embedding(input_dim=1000, output_dim=16, input_length=10),
  tf.keras.layers.Dense(1, activation='sigmoid')
])

medium

A. Dense layer activation should be 'relu' for binary output

B. Embedding layer input_dim should be output_dim

C. Missing Flatten or GlobalPooling layer before Dense

D. input_length parameter is not allowed in Embedding layer

Solution

1. Step 1: Analyze model layer sequence

   Embedding outputs 3D tensor (batch, seq_len, embed_dim). Dense expects 2D input, so flattening or pooling is needed.

2. Step 2: Identify missing layer

   Without Flatten or GlobalPooling, Dense layer will raise shape error.

3. Final Answer:

   Missing Flatten or GlobalPooling layer before Dense -> Option C

4. Quick Check:

   Embedding output must be flattened before Dense [OK]

Hint: Add Flatten or pooling before Dense after Embedding [OK]

Common Mistakes:

• Confusing input_dim and output_dim in Embedding
• Using wrong activation for binary classification
• Thinking input_length is invalid parameter

5. You want to improve a sentiment analysis model's accuracy on short reviews. Which approach is best?

hard

A. Increase embedding dimension and add a Bidirectional LSTM layer

B. Reduce embedding dimension and remove all recurrent layers

C. Use only a Dense layer without embedding

D. Train on fewer examples to avoid overfitting

Solution

1. Step 1: Consider model improvements for short text

   Increasing embedding dimension helps capture more word features; Bidirectional LSTM captures context from both directions.

2. Step 2: Evaluate other options

   Reducing embedding or removing recurrent layers loses information; training on fewer examples reduces learning quality.

3. Final Answer:

   Increase embedding dimension and add a Bidirectional LSTM layer -> Option A

4. Quick Check:

   More embedding + Bidirectional LSTM = better short text accuracy [OK]

Hint: Use bigger embeddings and bidirectional LSTM for short texts [OK]

Common Mistakes:

• Thinking smaller embeddings improve accuracy
• Removing recurrent layers harms context understanding
• Training on fewer examples reduces model learning