One-hot encoding for text in NLP - Model Metrics & Evaluation

Metrics & Evaluation - One-hot encoding for text

Which metric matters for One-hot encoding for text and WHY

One-hot encoding is a way to turn words into numbers so a computer can understand text. It does not create a model by itself but helps prepare data. When using one-hot encoded text in models, common metrics like accuracy, precision, and recall matter to check how well the model learns from this data. The choice depends on the task: for example, accuracy is good for balanced classes, while precision or recall matter more for imbalanced classes.

Confusion matrix example

Imagine a text classification model using one-hot encoded words to detect spam emails. Here is a confusion matrix from testing:

      | Predicted Spam | Predicted Not Spam |
      |----------------|--------------------|
      | True Spam: 40  | False Not Spam: 10 |
      | False Spam: 5  | True Not Spam: 45  |
    

Total samples = 40 + 10 + 5 + 45 = 100

Precision vs Recall tradeoff with examples

Precision tells us how many emails marked as spam really are spam. Recall tells us how many actual spam emails we found.

For spam detection, high precision means fewer good emails wrongly marked as spam (important to avoid losing important messages). High recall means catching most spam emails (important to keep inbox clean).

Sometimes improving precision lowers recall and vice versa. Choosing which to focus on depends on what is worse: missing spam or wrongly blocking good emails.

Good vs Bad metric values for one-hot encoded text models

Good values:

• Accuracy above 85% on balanced data
• Precision and recall both above 80% for important classes
• F1 score (balance of precision and recall) above 0.8

Bad values:

• Accuracy near 50% on balanced data (like guessing)
• Precision very low (many false alarms)
• Recall very low (many missed cases)
• F1 score below 0.5 showing poor balance

Common pitfalls in metrics for one-hot encoded text models

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced (e.g., 95% accuracy by always predicting the majority class).
• Data leakage: If test data leaks into training, metrics look better but model fails in real use.
• Overfitting: Very high training accuracy but low test accuracy means model memorizes training data, not generalizing well.
• Ignoring class imbalance: Metrics like accuracy hide poor performance on rare classes; use precision, recall, or F1 instead.

Self-check question

Your text classification model using one-hot encoding has 98% accuracy but only 12% recall on the spam class. Is it good for production? Why or why not?

Answer: No, it is not good. The model misses most spam emails (low recall), even though overall accuracy is high. This likely happens because spam is rare, so the model mostly predicts non-spam. For spam detection, missing spam is bad, so recall must improve.

Key Result

For one-hot encoded text models, balance precision and recall to ensure meaningful performance beyond simple accuracy.

Practice

1. What does one-hot encoding do to words in text processing?

easy

A. Converts each word into a vector with one 1 and rest 0s

B. Replaces words with their synonyms

C. Counts the number of letters in each word

D. Sorts words alphabetically

Solution

1. Step 1: Understand one-hot encoding concept

   One-hot encoding creates a vector for each word where only one position is 1 and all others are 0.

2. Step 2: Compare options with definition

   Only Converts each word into a vector with one 1 and rest 0s matches this definition exactly.

3. Final Answer:

   Converts each word into a vector with one 1 and rest 0s -> Option A

4. Quick Check:

   One-hot encoding = vector with single 1 [OK]

Hint: One-hot means one 1 in vector, rest zeros [OK]

Common Mistakes:

• Thinking it replaces words with synonyms
• Confusing with counting letters
• Assuming it sorts words

2. Which of the following is the correct Python syntax to create a one-hot vector for the word 'cat' from vocabulary ['cat', 'dog', 'bird']?

easy

A. one_hot = [0, 0, 1]

B. one_hot = [0, 1, 0]

C. one_hot = [1, 1, 0]

D. one_hot = [1, 0, 0]

Solution

1. Step 1: Identify the index of 'cat' in vocabulary

   'cat' is at index 0 in ['cat', 'dog', 'bird'].

2. Step 2: Create one-hot vector with 1 at index 0

   The vector should have 1 at position 0 and 0 elsewhere: [1, 0, 0].

3. Final Answer:

   [1, 0, 0] -> Option D

4. Quick Check:

   Index 0 gets 1 in one-hot vector [OK]

Hint: Index of word = position of 1 in vector [OK]

Common Mistakes:

• Putting 1 in wrong index
• Using multiple 1s in vector
• Confusing word order in vocabulary

3. What will be the output of this Python code?

vocab = ['apple', 'banana', 'cherry']
word = 'banana'
one_hot = [1 if w == word else 0 for w in vocab]
print(one_hot)

medium

A. [1, 0, 0]

B. [0, 1, 0]

C. [0, 0, 1]

D. [1, 1, 0]

Solution

1. Step 1: Understand list comprehension logic

   For each word in vocab, put 1 if it matches 'banana', else 0.

2. Step 2: Apply to vocab list

   'apple' != 'banana' -> 0, 'banana' == 'banana' -> 1, 'cherry' != 'banana' -> 0, so [0, 1, 0].

3. Final Answer:

   [0, 1, 0] -> Option B

4. Quick Check:

   Only 'banana' gets 1 in vector [OK]

Hint: Check which vocab word equals target word [OK]

Common Mistakes:

• Mixing up word positions
• Using 1 for all words
• Misreading list comprehension

4. Identify the error in this one-hot encoding code snippet:

vocab = ['red', 'green', 'blue']
word = 'green'
one_hot = [0 if w == word else 1 for w in vocab]
print(one_hot)

medium

A. The list comprehension syntax is invalid

B. The vocabulary list is missing a word

C. The condition is reversed; it should assign 1 when words match

D. The print statement syntax is incorrect

Solution

1. Step 1: Analyze the list comprehension condition

   It assigns 0 if word matches, else 1, which is opposite of one-hot logic.

2. Step 2: Correct logic for one-hot encoding

   One-hot should assign 1 when words match and 0 otherwise.

3. Final Answer:

   The condition is reversed; it should assign 1 when words match -> Option C

4. Quick Check:

   Match word -> 1, else 0 [OK]

Hint: One-hot sets 1 for match, not 0 [OK]

Common Mistakes:

• Reversing 0 and 1 in condition
• Assuming syntax error instead of logic error
• Ignoring correct vocabulary

5. Given a vocabulary ['sun', 'moon', 'star'] and a sentence 'moon star sun star', which one-hot encoded matrix correctly represents the sentence?

hard

A. [[0,1,0],[0,0,1],[1,0,0],[0,0,1]]

B. [[1,0,0],[0,1,0],[0,0,1],[0,1,0]]

C. [[0,0,1],[1,0,0],[0,1,0],[1,0,0]]

D. [[1,1,0],[0,0,1],[1,0,0],[0,0,1]]

Solution

1. Step 1: Map each word to its one-hot vector

   Vocabulary indices: 'sun'->0, 'moon'->1, 'star'->2. So 'moon'=[0,1,0], 'star'=[0,0,1], 'sun'=[1,0,0].

2. Step 2: Encode sentence words in order

   Sentence words: 'moon' -> [0,1,0], 'star' -> [0,0,1], 'sun' -> [1,0,0], 'star' -> [0,0,1].

3. Final Answer:

   [[0,1,0],[0,0,1],[1,0,0],[0,0,1]] -> Option A

4. Quick Check:

   Each word vector matches vocab index [OK]

Hint: Match word order and vocab index for vectors [OK]

Common Mistakes:

• Mixing word order in sentence
• Swapping indices of words
• Using vectors with multiple 1s