One-hot encoding for text in NLP - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is one-hot encoding in the context of text data?

One-hot encoding is a way to turn words into numbers by making a list where each word is represented by a vector with all zeros except for a single one in the position unique to that word.

beginner

Why do we use one-hot encoding for text in machine learning?

We use one-hot encoding to convert text into a format that computers can understand and work with, turning words into numbers without implying any order or similarity between them.

intermediate

What is a limitation of one-hot encoding for text?

One limitation is that one-hot encoding creates very large and sparse vectors when the vocabulary is big, which can be inefficient and does not capture the meaning or relationships between words.

intermediate

How does one-hot encoding handle new words not seen during training?

New words not in the original vocabulary cannot be represented directly by one-hot encoding, so they are often ignored or replaced with a special 'unknown' token vector.

beginner

Example: If the vocabulary is ['cat', 'dog', 'bird'], what is the one-hot vector for 'dog'?

The one-hot vector for 'dog' is [0, 1, 0] because 'dog' is the second word in the vocabulary list, so the second position is 1 and others are 0.

What does one-hot encoding do to a word in text data?

A Turns it into a vector with one 1 and rest 0s

B Assigns a random number to the word

C Replaces the word with its length

D Groups similar words together

One-hot encoding creates a vector where only one position is 1, representing the word uniquely.

Which problem can happen with one-hot encoding when vocabulary is very large?

A Words become similar

B Vectors become very small

C Words lose their order

D Vectors become sparse and large

One-hot vectors grow in size with vocabulary and mostly contain zeros, making them sparse and large.

How does one-hot encoding treat the relationship between words?

A Shows similarity between words

B Does not show any relationship

C Shows order of words

D Groups synonyms together

One-hot encoding treats each word independently without showing any relationship or similarity.

If the vocabulary is ['apple', 'banana', 'cherry'], what is the one-hot vector for 'cherry'?

A [1, 0, 0]

B [0, 1, 0]

C [0, 0, 1]

D [1, 1, 1]

'cherry' is the third word, so the third position is 1 and others are 0.

What happens if a new word not in the vocabulary appears during testing?

A It is ignored or replaced with an 'unknown' token

B It is assigned the vector of the closest word

C It gets a new one-hot vector automatically

D It causes an error

New words not in the vocabulary are usually replaced with a special 'unknown' token vector.

Explain what one-hot encoding is and why it is used for text data in machine learning.

Think about how computers need numbers instead of words.

Describe one limitation of one-hot encoding and how it affects text processing.

Consider what happens when you have many words.

Practice

1. What does one-hot encoding do to words in text processing?

easy

A. Converts each word into a vector with one 1 and rest 0s

B. Replaces words with their synonyms

C. Counts the number of letters in each word

D. Sorts words alphabetically

Solution

1. Step 1: Understand one-hot encoding concept

   One-hot encoding creates a vector for each word where only one position is 1 and all others are 0.

2. Step 2: Compare options with definition

   Only Converts each word into a vector with one 1 and rest 0s matches this definition exactly.

3. Final Answer:

   Converts each word into a vector with one 1 and rest 0s -> Option A

4. Quick Check:

   One-hot encoding = vector with single 1 [OK]

Hint: One-hot means one 1 in vector, rest zeros [OK]

Common Mistakes:

• Thinking it replaces words with synonyms
• Confusing with counting letters
• Assuming it sorts words

2. Which of the following is the correct Python syntax to create a one-hot vector for the word 'cat' from vocabulary ['cat', 'dog', 'bird']?

easy

A. one_hot = [0, 0, 1]

B. one_hot = [0, 1, 0]

C. one_hot = [1, 1, 0]

D. one_hot = [1, 0, 0]

Solution

1. Step 1: Identify the index of 'cat' in vocabulary

   'cat' is at index 0 in ['cat', 'dog', 'bird'].

2. Step 2: Create one-hot vector with 1 at index 0

   The vector should have 1 at position 0 and 0 elsewhere: [1, 0, 0].

3. Final Answer:

   [1, 0, 0] -> Option D

4. Quick Check:

   Index 0 gets 1 in one-hot vector [OK]

Hint: Index of word = position of 1 in vector [OK]

Common Mistakes:

• Putting 1 in wrong index
• Using multiple 1s in vector
• Confusing word order in vocabulary

3. What will be the output of this Python code?

vocab = ['apple', 'banana', 'cherry']
word = 'banana'
one_hot = [1 if w == word else 0 for w in vocab]
print(one_hot)

medium

A. [1, 0, 0]

B. [0, 1, 0]

C. [0, 0, 1]

D. [1, 1, 0]

Solution

1. Step 1: Understand list comprehension logic

   For each word in vocab, put 1 if it matches 'banana', else 0.

2. Step 2: Apply to vocab list

   'apple' != 'banana' -> 0, 'banana' == 'banana' -> 1, 'cherry' != 'banana' -> 0, so [0, 1, 0].

3. Final Answer:

   [0, 1, 0] -> Option B

4. Quick Check:

   Only 'banana' gets 1 in vector [OK]

Hint: Check which vocab word equals target word [OK]

Common Mistakes:

• Mixing up word positions
• Using 1 for all words
• Misreading list comprehension

4. Identify the error in this one-hot encoding code snippet:

vocab = ['red', 'green', 'blue']
word = 'green'
one_hot = [0 if w == word else 1 for w in vocab]
print(one_hot)

medium

A. The list comprehension syntax is invalid

B. The vocabulary list is missing a word

C. The condition is reversed; it should assign 1 when words match

D. The print statement syntax is incorrect

Solution

1. Step 1: Analyze the list comprehension condition

   It assigns 0 if word matches, else 1, which is opposite of one-hot logic.

2. Step 2: Correct logic for one-hot encoding

   One-hot should assign 1 when words match and 0 otherwise.

3. Final Answer:

   The condition is reversed; it should assign 1 when words match -> Option C

4. Quick Check:

   Match word -> 1, else 0 [OK]

Hint: One-hot sets 1 for match, not 0 [OK]

Common Mistakes:

• Reversing 0 and 1 in condition
• Assuming syntax error instead of logic error
• Ignoring correct vocabulary

5. Given a vocabulary ['sun', 'moon', 'star'] and a sentence 'moon star sun star', which one-hot encoded matrix correctly represents the sentence?

hard

A. [[0,1,0],[0,0,1],[1,0,0],[0,0,1]]

B. [[1,0,0],[0,1,0],[0,0,1],[0,1,0]]

C. [[0,0,1],[1,0,0],[0,1,0],[1,0,0]]

D. [[1,1,0],[0,0,1],[1,0,0],[0,0,1]]

Solution

1. Step 1: Map each word to its one-hot vector

   Vocabulary indices: 'sun'->0, 'moon'->1, 'star'->2. So 'moon'=[0,1,0], 'star'=[0,0,1], 'sun'=[1,0,0].

2. Step 2: Encode sentence words in order

   Sentence words: 'moon' -> [0,1,0], 'star' -> [0,0,1], 'sun' -> [1,0,0], 'star' -> [0,0,1].

3. Final Answer:

   [[0,1,0],[0,0,1],[1,0,0],[0,0,1]] -> Option A

4. Quick Check:

   Each word vector matches vocab index [OK]

Hint: Match word order and vocab index for vectors [OK]

Common Mistakes:

• Mixing word order in sentence
• Swapping indices of words
• Using vectors with multiple 1s