What NLP actually does - ML Experiment: Train & Evaluate

Experiment - What NLP actually does

Problem:We want to teach a computer to understand simple sentences and classify their meaning, like telling if a sentence is happy or sad.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%

Issue:The model is overfitting: it learns the training data too well but does not generalize to new sentences.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85%, while keeping training accuracy below 92%.

You can only change the model architecture or training settings.

Do not add more training data.

Keep the input data and labels the same.

Hint 1

Hint 2

Hint 3

Solution

NLP

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Sample data placeholders (replace with actual data)
X_train = ...  # tokenized and padded training sentences
y_train = ...  # training labels
X_val = ...    # tokenized and padded validation sentences
y_val = ...    # validation labels

vocab_size = 10000
embedding_dim = 64
max_length = 100

model = Sequential([
    Embedding(vocab_size, embedding_dim, input_length=max_length),
    LSTM(64, return_sequences=False),
    Dropout(0.5),
    Dense(32, activation='relu'),
    Dropout(0.5),
    Dense(1, activation='sigmoid')
])

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)

history = model.fit(
    X_train, y_train,
    epochs=20,
    batch_size=32,
    validation_data=(X_val, y_val),
    callbacks=[early_stop]
)

Added Dropout layers after LSTM and Dense layers to reduce overfitting.

Implemented EarlyStopping to stop training when validation loss stops improving.

Kept model size moderate with 64 LSTM units and 32 Dense units.

Results Interpretation

Before: Training accuracy was 95%, but validation accuracy was only 70%, showing overfitting.

After: Training accuracy reduced to 90%, and validation accuracy improved to 87%, showing better generalization.

Adding dropout and early stopping helps the model not memorize training data too much, so it performs better on new sentences. This is how NLP models learn to understand language more reliably.

Bonus Experiment

Try using a simpler model like a GRU instead of LSTM or reduce the embedding size to see if it improves validation accuracy further.

💡 Hint

GRU layers are faster and sometimes less prone to overfitting. Smaller embeddings reduce model complexity.

Practice

1. What is the main goal of Natural Language Processing (NLP)?

easy

A. To help computers understand and work with human language

B. To create images from text descriptions

C. To speed up computer hardware

D. To store large amounts of data efficiently

Solution

1. Step 1: Understand NLP's purpose

   NLP focuses on making computers understand human language, like speech or text.

2. Step 2: Compare options

   Only To help computers understand and work with human language describes this goal; others are unrelated to language understanding.

3. Final Answer:

   To help computers understand and work with human language -> Option A

4. Quick Check:

   NLP goal = Understand human language [OK]

Hint: NLP = computers understanding human language [OK]

Common Mistakes:

• Confusing NLP with image processing
• Thinking NLP is about hardware or storage
• Mixing NLP with unrelated computer tasks

2. Which of the following is a correct step in basic NLP processing?

easy

A. Compiling code into machine language

B. Splitting text into words or sentences

C. Encrypting data for security

D. Formatting images for display

Solution

1. Step 1: Identify NLP preprocessing steps

   Basic NLP starts by breaking text into smaller parts like words or sentences.

2. Step 2: Eliminate unrelated options

   Options B, C, and D relate to programming, security, or images, not NLP text processing.

3. Final Answer:

   Splitting text into words or sentences -> Option B

4. Quick Check:

   Basic NLP step = Text splitting [OK]

Hint: NLP starts by breaking text into pieces [OK]

Common Mistakes:

• Confusing NLP steps with programming tasks
• Mixing text processing with encryption or image tasks
• Choosing unrelated computer operations

3. Given this Python code using NLP, what will be the output?

import nltk
text = "Hello world!"
tokens = nltk.word_tokenize(text)
print(tokens)

medium

A. ['Hello world!']

B. Error: nltk module not found

C. ['Hello_world!']

D. ['Hello', 'world', '!']

Solution

1. Step 1: Understand nltk.word_tokenize function

   This function splits text into words and punctuation marks as separate tokens.

2. Step 2: Apply tokenization to the text

   "Hello world!" becomes ['Hello', 'world', '!'] as separate tokens.

3. Final Answer:

   ['Hello', 'world', '!'] -> Option D

4. Quick Check:

   Tokenize "Hello world!" = ['Hello', 'world', '!'] [OK]

Hint: Tokenize splits words and punctuation separately [OK]

Common Mistakes:

• Expecting the whole sentence as one token
• Ignoring punctuation as separate tokens
• Assuming code will error without nltk installed

4. Find the error in this NLP code snippet:

text = "I love NLP!"
tokens = text.split()
print(tokens.lower())

medium

A. Calling lower() on a list instead of a string

B. Using split() instead of word_tokenize()

C. Missing import statement for nltk

D. No error, code runs fine

Solution

1. Step 1: Analyze the code operations

   text.split() returns a list of words, but tokens.lower() tries to call lower() on a list.

2. Step 2: Identify the error type

   Lists do not have a lower() method, causing an AttributeError.

3. Final Answer:

   Calling lower() on a list instead of a string -> Option A

4. Quick Check:

   lower() on list causes error [OK]

Hint: lower() works on strings, not lists [OK]

Common Mistakes:

• Thinking split() is wrong here
• Ignoring that lower() is called on a list
• Assuming code runs without error

5. You want to build a chatbot that understands user questions and answers them. Which NLP steps should you include?

hard

A. Database indexing, query optimization, and caching

B. Image resizing, color correction, and pixel filtering

C. Tokenization, part-of-speech tagging, named entity recognition, and intent detection

D. Hardware acceleration, memory management, and threading

Solution

1. Step 1: Identify NLP tasks for chatbot understanding

   Tokenization breaks text into words, POS tagging finds word roles, named entity recognition finds names, and intent detection understands user goals.

2. Step 2: Eliminate unrelated options

   Options A, B, and D relate to databases, images, or hardware, not language understanding.

3. Final Answer:

   Tokenization, part-of-speech tagging, named entity recognition, and intent detection -> Option C

4. Quick Check:

   Chatbot NLP steps = Tokenize + Tag + Recognize + Detect intent [OK]

Hint: Chatbots need tokenizing, tagging, recognizing, and intent detection [OK]

Common Mistakes:

• Confusing NLP with image or hardware tasks
• Ignoring intent detection for understanding
• Choosing unrelated computer processes