Bag of Words and TF-IDF in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding TF-IDF Importance

Which statement best explains why TF-IDF is preferred over simple Bag of Words for text classification?

A TF-IDF reduces the impact of common words by weighting them lower, highlighting important words unique to documents.

B TF-IDF converts text into images for better visual classification.

C TF-IDF removes all stop words from the text before analysis.

D TF-IDF counts the total number of words in a document to improve classification accuracy.

💡 Hint

Think about how common words like 'the' or 'and' affect simple word counts.

TF-IDF lowers the weight of very common words across all documents, so unique and important words stand out more for classification.

❓ Predict Output

intermediate

Output of Bag of Words Vectorization

What is the output of the following Python code using CountVectorizer from sklearn?

ML Python

from sklearn.feature_extraction.text import CountVectorizer
corpus = ['apple banana apple', 'banana orange', 'apple orange orange']
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(corpus)
print(X.toarray())

A

[[2 0 1]
 [1 1 0]
 [0 2 1]]

B

[[1 2 0]
 [0 1 1]
 [2 0 1]]

C

[[1 1 1]
 [1 1 1]
 [1 1 1]]

D

[[2 1 0]
 [0 1 1]
 [1 0 2]]

💡 Hint

CountVectorizer orders words alphabetically and counts their occurrences per document.

The words are ordered as ['apple', 'banana', 'orange']. The counts per document match the output in option D.

❓ Model Choice

advanced

Choosing Vectorization for Sparse Data

You have a large collection of short text messages with many unique words appearing rarely. Which vectorization method is best to reduce noise and improve model performance?

A Use TF-IDF to weight words by importance across messages.

B Use Bag of Words with raw counts only.

C Use one-hot encoding for each word.

D Use word embeddings without any weighting.

💡 Hint

Consider how to reduce the effect of rare or common words in sparse text data.

TF-IDF weights words by their importance, reducing noise from rare or overly common words, which helps models learn better from sparse data.

❓ Metrics

advanced

Evaluating Text Vectorization Impact

After applying Bag of Words and TF-IDF vectorization separately on the same dataset, you train a classifier. Which metric difference best indicates TF-IDF improved the model?

A Lower accuracy but higher training time with TF-IDF.

B Higher accuracy and lower false positive rate with TF-IDF.

C Same accuracy but higher false negative rate with TF-IDF.

D Higher accuracy but higher false positive rate with Bag of Words.

💡 Hint

Better vectorization should improve correct predictions and reduce mistakes.

Improved vectorization like TF-IDF usually leads to higher accuracy and fewer false positives, showing better model performance.

🔧 Debug

expert

Debugging TF-IDF Vectorizer Output

Given this code snippet, what error or issue will occur?

ML Python

from sklearn.feature_extraction.text import TfidfVectorizer
corpus = ['cat dog', 'dog mouse', 'cat mouse mouse']
vectorizer = TfidfVectorizer(stop_words=['dog'])
X = vectorizer.fit_transform(corpus)
print(vectorizer.get_feature_names_out())

A Output will include 'dog' because stop_words parameter is ignored.

B SyntaxError due to incorrect stop_words parameter type.

C Output will be ['cat', 'mouse'] because 'dog' is removed as a stop word.

D ValueError because stop_words must be a string, not a list.

💡 Hint

Check how TfidfVectorizer accepts stop words as input.

TfidfVectorizer accepts a list of stop words and removes them from the vocabulary, so 'dog' is excluded.