One-hot encoding for text in NLP - ML Experiment: Train & Evaluate

Experiment - One-hot encoding for text

Problem:You want to convert a list of text sentences into one-hot encoded vectors to prepare data for a machine learning model.

Current Metrics:N/A - currently, the text is raw and not encoded, so the model cannot train.

Issue:The text data is not in a numeric format that machine learning models can understand. Without encoding, the model cannot learn from the text.

Your Task

Convert the given list of sentences into one-hot encoded vectors using a vocabulary built from the text. Verify the encoding by printing the one-hot vectors.

Use only Python standard libraries and scikit-learn.

Do not use embedding layers or other complex encodings.

Keep the vocabulary size manageable by using the unique words from the input sentences.

Hint 1

Hint 2

Hint 3

Solution

NLP

from sklearn.feature_extraction.text import CountVectorizer

# Sample text data
sentences = [
    "I love machine learning",
    "Machine learning is fun",
    "I love coding",
    "Coding is fun"
]

# Create CountVectorizer with binary=True for one-hot encoding
vectorizer = CountVectorizer(binary=True)

# Fit on sentences to build vocabulary
vectorizer.fit(sentences)

# Transform sentences to one-hot encoded vectors
one_hot_vectors = vectorizer.transform(sentences)

# Convert sparse matrix to array for display
one_hot_array = one_hot_vectors.toarray()

# Print vocabulary and one-hot vectors
print("Vocabulary:", vectorizer.get_feature_names_out())
print("One-hot encoded vectors:")
for sentence, vector in zip(sentences, one_hot_array):
    print(f"{sentence}: {vector}")

Used CountVectorizer with binary=True to create one-hot encoding.

Built vocabulary from the input sentences.

Transformed sentences into one-hot encoded vectors.

Printed vocabulary and vectors to verify encoding.

Results Interpretation

Before encoding, the text was raw and unusable by ML models.

After encoding, each sentence is represented as a vector of 0s and 1s indicating presence of words.

This numeric format can now be fed into ML models.

One-hot encoding converts text into a simple numeric format by marking which words appear in each sentence. This is a basic but important step to prepare text data for machine learning.

Bonus Experiment

Try using TF-IDF encoding instead of one-hot encoding to see how word importance affects the vectors.

💡 Hint

Use sklearn's TfidfVectorizer and compare the output vectors to one-hot vectors.

Practice

1. What does one-hot encoding do to words in text processing?

easy

A. Converts each word into a vector with one 1 and rest 0s

B. Replaces words with their synonyms

C. Counts the number of letters in each word

D. Sorts words alphabetically

Solution

1. Step 1: Understand one-hot encoding concept

   One-hot encoding creates a vector for each word where only one position is 1 and all others are 0.

2. Step 2: Compare options with definition

   Only Converts each word into a vector with one 1 and rest 0s matches this definition exactly.

3. Final Answer:

   Converts each word into a vector with one 1 and rest 0s -> Option A

4. Quick Check:

   One-hot encoding = vector with single 1 [OK]

Hint: One-hot means one 1 in vector, rest zeros [OK]

Common Mistakes:

• Thinking it replaces words with synonyms
• Confusing with counting letters
• Assuming it sorts words

2. Which of the following is the correct Python syntax to create a one-hot vector for the word 'cat' from vocabulary ['cat', 'dog', 'bird']?

easy

A. one_hot = [0, 0, 1]

B. one_hot = [0, 1, 0]

C. one_hot = [1, 1, 0]

D. one_hot = [1, 0, 0]

Solution

1. Step 1: Identify the index of 'cat' in vocabulary

   'cat' is at index 0 in ['cat', 'dog', 'bird'].

2. Step 2: Create one-hot vector with 1 at index 0

   The vector should have 1 at position 0 and 0 elsewhere: [1, 0, 0].

3. Final Answer:

   [1, 0, 0] -> Option D

4. Quick Check:

   Index 0 gets 1 in one-hot vector [OK]

Hint: Index of word = position of 1 in vector [OK]

Common Mistakes:

• Putting 1 in wrong index
• Using multiple 1s in vector
• Confusing word order in vocabulary

3. What will be the output of this Python code?

vocab = ['apple', 'banana', 'cherry']
word = 'banana'
one_hot = [1 if w == word else 0 for w in vocab]
print(one_hot)

medium

A. [1, 0, 0]

B. [0, 1, 0]

C. [0, 0, 1]

D. [1, 1, 0]

Solution

1. Step 1: Understand list comprehension logic

   For each word in vocab, put 1 if it matches 'banana', else 0.

2. Step 2: Apply to vocab list

   'apple' != 'banana' -> 0, 'banana' == 'banana' -> 1, 'cherry' != 'banana' -> 0, so [0, 1, 0].

3. Final Answer:

   [0, 1, 0] -> Option B

4. Quick Check:

   Only 'banana' gets 1 in vector [OK]

Hint: Check which vocab word equals target word [OK]

Common Mistakes:

• Mixing up word positions
• Using 1 for all words
• Misreading list comprehension

4. Identify the error in this one-hot encoding code snippet:

vocab = ['red', 'green', 'blue']
word = 'green'
one_hot = [0 if w == word else 1 for w in vocab]
print(one_hot)

medium

A. The list comprehension syntax is invalid

B. The vocabulary list is missing a word

C. The condition is reversed; it should assign 1 when words match

D. The print statement syntax is incorrect

Solution

1. Step 1: Analyze the list comprehension condition

   It assigns 0 if word matches, else 1, which is opposite of one-hot logic.

2. Step 2: Correct logic for one-hot encoding

   One-hot should assign 1 when words match and 0 otherwise.

3. Final Answer:

   The condition is reversed; it should assign 1 when words match -> Option C

4. Quick Check:

   Match word -> 1, else 0 [OK]

Hint: One-hot sets 1 for match, not 0 [OK]

Common Mistakes:

• Reversing 0 and 1 in condition
• Assuming syntax error instead of logic error
• Ignoring correct vocabulary

5. Given a vocabulary ['sun', 'moon', 'star'] and a sentence 'moon star sun star', which one-hot encoded matrix correctly represents the sentence?

hard

A. [[0,1,0],[0,0,1],[1,0,0],[0,0,1]]

B. [[1,0,0],[0,1,0],[0,0,1],[0,1,0]]

C. [[0,0,1],[1,0,0],[0,1,0],[1,0,0]]

D. [[1,1,0],[0,0,1],[1,0,0],[0,0,1]]

Solution

1. Step 1: Map each word to its one-hot vector

   Vocabulary indices: 'sun'->0, 'moon'->1, 'star'->2. So 'moon'=[0,1,0], 'star'=[0,0,1], 'sun'=[1,0,0].

2. Step 2: Encode sentence words in order

   Sentence words: 'moon' -> [0,1,0], 'star' -> [0,0,1], 'sun' -> [1,0,0], 'star' -> [0,0,1].

3. Final Answer:

   [[0,1,0],[0,0,1],[1,0,0],[0,0,1]] -> Option A

4. Quick Check:

   Each word vector matches vocab index [OK]

Hint: Match word order and vocab index for vectors [OK]

Common Mistakes:

• Mixing word order in sentence
• Swapping indices of words
• Using vectors with multiple 1s