Regular expressions for text cleaning in NLP - Model Metrics & Evaluation

Metrics & Evaluation - Regular expressions for text cleaning

Which metric matters for this concept and WHY

When using regular expressions for text cleaning, the key metric is data quality improvement. This means how well the cleaning removes unwanted characters or patterns without losing important information. Metrics like precision and recall can be adapted here:

• Precision: How many of the removed parts were actually unwanted (true removals vs wrong removals).
• Recall: How many unwanted parts were successfully removed (true removals vs missed unwanted parts).

Good cleaning keeps useful text intact (high precision) and removes all noise (high recall). This helps models learn better from clean data.

Confusion matrix or equivalent visualization (ASCII)

Unwanted parts:  | Removed | Not Removed
-----------------|---------|------------
Actually Removed |   TP    |    FN      
Actually Kept    |   FP    |    TN      

TP = Unwanted parts correctly removed
FP = Useful parts wrongly removed
FN = Unwanted parts missed
TN = Useful parts kept
    

Precision vs Recall tradeoff with concrete examples

Imagine cleaning tweets:

• High precision, low recall: You only remove very obvious noise like URLs, but miss emojis or hashtags. You keep most useful text but some noise remains.
• High recall, low precision: You remove all special characters including some words or emojis that carry meaning. You get rid of noise but lose useful info.

Balance is key: remove enough noise to help the model but keep important text for learning.

What "good" vs "bad" metric values look like for this use case

• Good cleaning: Precision > 0.9 and Recall > 0.85. Most noise removed, very few useful parts lost.
• Bad cleaning: Precision < 0.7 or Recall < 0.5. Either too much useful text removed or too much noise left.

Good cleaning leads to better model accuracy and faster training.

Metrics pitfalls

• Accuracy paradox: Simply counting how many characters removed is misleading. Removing too much can look like high "accuracy" but harms data quality.
• Data leakage: Over-cleaning can remove important signals that models need, causing poor generalization.
• Overfitting indicators: If cleaning is too strict on training data patterns, model may fail on new text with different noise.

Self-check question

Your text cleaning removes 98% of unwanted noise but also removes 30% of useful words (low precision). Is this good?

Answer: No, because losing 30% of useful words means the model will miss important information. You should improve precision to keep useful text while still removing noise.

Key Result

Effective text cleaning balances high precision and recall to remove noise while preserving useful text.