Stopword removal in NLP - Model Metrics & Evaluation

Metrics & Evaluation - Stopword removal

Which metric matters for Stopword removal and WHY

Stopword removal is a preprocessing step in text analysis. It helps clean text by removing common words like "the" or "and" that do not add meaning. The main goal is to improve the quality of features for models.

Metrics to check here are impact on downstream model performance, such as accuracy or F1 score of a text classifier. We want to see if removing stopwords helps the model understand text better.

Also, check vocabulary size reduction and processing speed. Removing stopwords should reduce text size and speed up training without losing important information.

Confusion matrix or equivalent visualization

Stopword removal itself does not produce a confusion matrix. But after removing stopwords, you train a model and get a confusion matrix like this:

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |   TP=80  |  FN=20  
      Negative           |   FP=10  |  TN=90  
    

This shows how well the model performs after stopword removal. Compare it to a model trained without stopword removal to see if metrics improve.

Precision vs Recall tradeoff with concrete examples

Removing stopwords can affect precision and recall differently:

• Precision measures how many predicted positive texts are truly positive. If stopword removal removes important words, precision may drop.
• Recall measures how many actual positive texts are found. If stopwords hide key signals, recall may drop.

Example: In spam detection, removing stopwords might remove words that help spot spam. This could lower recall (missing spam). But it might also reduce noise and improve precision (fewer false spam alerts).

So, test both metrics to find the best balance for your task.

What "good" vs "bad" metric values look like for Stopword removal

Good:

• Model accuracy or F1 score improves or stays the same after stopword removal.
• Vocabulary size reduces significantly, speeding up training.
• Precision and recall remain balanced or improve.

Bad:

• Model accuracy or F1 score drops noticeably.
• Precision or recall drops sharply, meaning important info was lost.
• Vocabulary size does not reduce much, so no speed benefit.

Metrics pitfalls

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced. Always check precision and recall.
• Data leakage: If stopword lists are created using test data, results will be too optimistic.
• Overfitting indicators: If model performs well on training but poorly on test data after stopword removal, it may have lost important signals.
• Removing too many words: Aggressive stopword removal can remove meaningful words, hurting model performance.

Self-check question

Your text classifier has 98% accuracy but only 12% recall on the positive class after stopword removal. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall means the model misses most positive cases, which can be critical depending on the task. High accuracy alone is misleading if the positive class is rare. You should improve recall before using the model.

Key Result

Stopword removal should improve or maintain model accuracy and F1 while reducing vocabulary size; watch for drops in recall or precision.

Practice

1. What is the main purpose of stopword removal in natural language processing?

easy

A. To correct spelling mistakes in text

B. To translate text into another language

C. To count the number of words in a sentence

D. To remove common words that do not add much meaning

Solution

1. Step 1: Understand what stopwords are

   Stopwords are common words like 'the', 'is', 'and' that usually don't add important meaning.

2. Step 2: Identify the purpose of removing stopwords

   Removing these words helps focus on meaningful words for better analysis.

3. Final Answer:

   To remove common words that do not add much meaning -> Option D

4. Quick Check:

   Stopword removal = Remove common meaningless words [OK]

Hint: Stopwords are common filler words removed to focus on meaning [OK]

Common Mistakes:

• Thinking stopword removal translates text
• Confusing stopword removal with spell checking
• Believing it counts words instead of removing them

2. Which of the following Python code snippets correctly removes stopwords from a list of words using NLTK?

easy

A. filtered_words = [w for w in words if w not in stopwords.words('english')]

B. filtered_words = [w for w in words if w in stopwords.words('english')]

C. filtered_words = stopwords.remove(words)

D. filtered_words = words.remove(stopwords.words('english'))

Solution

1. Step 1: Understand NLTK stopword removal syntax

   We keep words that are NOT in the stopwords list using a list comprehension.

2. Step 2: Check each option

   filtered_words = [w for w in words if w not in stopwords.words('english')] correctly filters out stopwords. filtered_words = [w for w in words if w in stopwords.words('english')] keeps only stopwords, which is wrong. Options C and D use invalid methods.

3. Final Answer:

   filtered_words = [w for w in words if w not in stopwords.words('english')] -> Option A

4. Quick Check:

   Keep words not in stopwords list = filtered_words = [w for w in words if w not in stopwords.words('english')] [OK]

Hint: Filter words not in stopwords list using list comprehension [OK]

Common Mistakes:

• Using 'in' instead of 'not in' to filter stopwords
• Calling non-existent methods like stopwords.remove()
• Confusing filtering logic to keep stopwords instead of removing

3. Given the code below, what is the output?

import nltk
from nltk.corpus import stopwords
nltk.download('stopwords')
words = ['this', 'is', 'a', 'test']
filtered = [w for w in words if w not in stopwords.words('english')]
print(filtered)

medium

A. ['this', 'test']

B. ['this', 'is', 'a', 'test']

C. ['test']

D. []

Solution

1. Step 1: Identify stopwords in the list

   Stopwords in English include 'this', 'is', 'a'. 'test' is not a stopword.

2. Step 2: Filter out stopwords

   The list comprehension removes 'this', 'is', 'a', leaving only 'test'.

3. Final Answer:

   ['test'] -> Option C

4. Quick Check:

   Only non-stopword 'test' remains [OK]

Hint: Remove common words; only meaningful words remain [OK]

Common Mistakes:

• Assuming all words remain after removal
• Forgetting to download stopwords corpus
• Confusing which words are stopwords

4. The following code is intended to remove stopwords from a list of words, but it raises an error. What is the problem?

from nltk.corpus import stopwords
words = ['hello', 'world']
filtered = [w for w in words if w not in stopwords('english')]
print(filtered)

medium

A. stopwords is not a function; should use stopwords.words('english')

B. The list comprehension syntax is incorrect

C. The variable 'words' is not defined

D. The print statement is missing parentheses

Solution

1. Step 1: Check how stopwords are accessed

   stopwords is a module, and stopwords.words('english') returns the list of stopwords.

2. Step 2: Identify the error in code

   The code calls stopwords('english'), which is invalid and causes an error.

3. Final Answer:

   stopwords is not a function; should use stopwords.words('english') -> Option A

4. Quick Check:

   Use stopwords.words('english') to get stopwords list [OK]

Hint: Use stopwords.words('english'), not stopwords('english') [OK]

Common Mistakes:

• Calling stopwords as a function instead of accessing .words()
• Misunderstanding list comprehension syntax
• Assuming print needs no parentheses in Python 3

5. You want to remove stopwords from a text but keep the word 'not' because it changes meaning. How can you modify the stopword list in NLTK to do this?

hard

A. Add 'not' to the stopwords list before filtering

B. Remove 'not' from the stopwords list before filtering

C. Replace 'not' with a synonym before filtering

D. Ignore stopword removal and keep all words

Solution

1. Step 1: Understand default stopwords list

   NLTK's stopwords list includes 'not', which would be removed by default.

2. Step 2: Modify stopwords list to keep 'not'

   Remove 'not' from the stopwords list before filtering to keep it in the text.

3. Final Answer:

   Remove 'not' from the stopwords list before filtering -> Option B

4. Quick Check:

   Modify stopwords list to keep important words [OK]

Hint: Delete 'not' from stopwords list to keep it in text [OK]

Common Mistakes:

• Adding 'not' to stopwords instead of removing
• Replacing words instead of modifying stopwords
• Skipping stopword removal entirely