Bag of Words and TF-IDF in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding TF-IDF Importance

Which statement best explains why TF-IDF is preferred over simple Bag of Words for text classification?

A TF-IDF reduces the impact of common words by weighting them lower, highlighting important words unique to documents.

B TF-IDF converts text into images for better visual classification.

C TF-IDF removes all stop words from the text before analysis.

D TF-IDF counts the total number of words in a document to improve classification accuracy.

💡 Hint

Think about how common words like 'the' or 'and' affect simple word counts.

TF-IDF lowers the weight of very common words across all documents, so unique and important words stand out more for classification.

❓ Predict Output

intermediate

Output of Bag of Words Vectorization

What is the output of the following Python code using CountVectorizer from sklearn?

ML Python

from sklearn.feature_extraction.text import CountVectorizer
corpus = ['apple banana apple', 'banana orange', 'apple orange orange']
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(corpus)
print(X.toarray())

A

[[2 0 1]
 [1 1 0]
 [0 2 1]]

B

[[1 2 0]
 [0 1 1]
 [2 0 1]]

C

[[1 1 1]
 [1 1 1]
 [1 1 1]]

D

[[2 1 0]
 [0 1 1]
 [1 0 2]]

💡 Hint

CountVectorizer orders words alphabetically and counts their occurrences per document.

The words are ordered as ['apple', 'banana', 'orange']. The counts per document match the output in option D.

❓ Model Choice

advanced

Choosing Vectorization for Sparse Data

You have a large collection of short text messages with many unique words appearing rarely. Which vectorization method is best to reduce noise and improve model performance?

A Use TF-IDF to weight words by importance across messages.

B Use Bag of Words with raw counts only.

C Use one-hot encoding for each word.

D Use word embeddings without any weighting.

💡 Hint

Consider how to reduce the effect of rare or common words in sparse text data.

TF-IDF weights words by their importance, reducing noise from rare or overly common words, which helps models learn better from sparse data.

❓ Metrics

advanced

Evaluating Text Vectorization Impact

After applying Bag of Words and TF-IDF vectorization separately on the same dataset, you train a classifier. Which metric difference best indicates TF-IDF improved the model?

A Lower accuracy but higher training time with TF-IDF.

B Higher accuracy and lower false positive rate with TF-IDF.

C Same accuracy but higher false negative rate with TF-IDF.

D Higher accuracy but higher false positive rate with Bag of Words.

💡 Hint

Better vectorization should improve correct predictions and reduce mistakes.

Improved vectorization like TF-IDF usually leads to higher accuracy and fewer false positives, showing better model performance.

🔧 Debug

expert

Debugging TF-IDF Vectorizer Output

Given this code snippet, what error or issue will occur?

ML Python

from sklearn.feature_extraction.text import TfidfVectorizer
corpus = ['cat dog', 'dog mouse', 'cat mouse mouse']
vectorizer = TfidfVectorizer(stop_words=['dog'])
X = vectorizer.fit_transform(corpus)
print(vectorizer.get_feature_names_out())

A Output will include 'dog' because stop_words parameter is ignored.

B SyntaxError due to incorrect stop_words parameter type.

C Output will be ['cat', 'mouse'] because 'dog' is removed as a stop word.

D ValueError because stop_words must be a string, not a list.

💡 Hint

Check how TfidfVectorizer accepts stop words as input.

TfidfVectorizer accepts a list of stop words and removes them from the vocabulary, so 'dog' is excluded.

Practice

1. What does the Bag of Words model primarily count in a text document?

easy

A. The order of words in the document

B. The frequency of each word appearing in the document

C. The sentiment of the document

D. The length of the document in characters

Solution

1. Step 1: Understand Bag of Words concept

   Bag of Words counts how many times each word appears, ignoring order.

2. Step 2: Identify what is counted

   It counts word frequency, not order, sentiment, or length.

3. Final Answer:

   The frequency of each word appearing in the document -> Option B

4. Quick Check:

   Bag of Words = word counts [OK]

Hint: Bag of Words counts words, not order or meaning [OK]

Common Mistakes:

• Thinking Bag of Words considers word order
• Confusing frequency with sentiment
• Assuming it measures document length

2. Which of the following is the correct way to create a Bag of Words vector using scikit-learn's CountVectorizer in Python?

easy

A. from sklearn.feature_extraction.text import CountVectorizer
vectorizer = CountVectorizer()
matrix = vectorizer.fit(documents)

B. from sklearn.feature_extraction.text import TfidfVectorizer
vectorizer = CountVectorizer()
matrix = vectorizer.fit_transform(documents)

C. from sklearn.feature_extraction.text import CountVectorizer
vectorizer = CountVectorizer()
matrix = vectorizer.transform(documents)

D. from sklearn.feature_extraction.text import CountVectorizer
vectorizer = CountVectorizer()
matrix = vectorizer.fit_transform(documents)

Solution

1. Step 1: Import and initialize CountVectorizer

   Correct import is from sklearn.feature_extraction.text import CountVectorizer and then create an instance.

2. Step 2: Fit and transform documents

   Use fit_transform to learn vocabulary and create the matrix in one step.

3. Final Answer:

   from sklearn.feature_extraction.text import CountVectorizer vectorizer = CountVectorizer() matrix = vectorizer.fit_transform(documents) -> Option D

4. Quick Check:

   CountVectorizer + fit_transform = Bag of Words matrix [OK]

Hint: Use fit_transform to build and apply vectorizer in one step [OK]

Common Mistakes:

• Using transform without fit first
• Importing TfidfVectorizer instead of CountVectorizer
• Calling fit without transform or fit_transform

3. Given the following Python code using TF-IDF, what will be the shape of the output matrix?

from sklearn.feature_extraction.text import TfidfVectorizer
docs = ['cat dog', 'dog mouse', 'cat mouse mouse']
vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(docs)
print(X.shape)

medium

A. (4, 3)

B. (3, 4)

C. (3, 3)

D. (3, 2)

Solution

1. Step 1: Count unique words across all documents

   The words are 'cat', 'dog', 'mouse'. 'mouse' appears twice but counts once. So unique words = 3.

2. Step 2: Check if TfidfVectorizer adds extra tokens

   TfidfVectorizer by default tokenizes and counts unique words. Here, it finds 3 unique words: 'cat', 'dog', 'mouse'. So matrix shape is (number of docs, number of unique words) = (3,3).

3. Step 3: Verify options

   (3, 3) is (3,3), matching our count. But (3, 4) is (3,4), which suggests 4 unique words. Let's check if any extra token is created.

4. Step 4: Check for default token pattern

   By default, TfidfVectorizer splits on word boundaries, so no extra tokens. So shape is (3,3).

5. Final Answer:

   (3, 3) -> Option C

6. Quick Check:

   Docs=3, unique words=3, shape=(3,3) [OK]

Hint: Shape = (documents, unique words) counted by vectorizer [OK]

Common Mistakes:

• Counting repeated words multiple times
• Confusing rows and columns in shape
• Assuming extra tokens are added automatically

4. The following code tries to create a TF-IDF matrix but raises an error. What is the main problem?

from sklearn.feature_extraction.text import TfidfVectorizer
vectorizer = TfidfVectorizer()
docs = ['apple orange', 'banana apple']
X = vectorizer.transform(docs)
print(X.toarray())

medium

A. fit_transform should be used before transform to learn vocabulary

B. The documents list is empty

C. TfidfVectorizer cannot handle multiple documents

D. toarray() is not a valid method for sparse matrices

Solution

1. Step 1: Understand TfidfVectorizer usage

   Before calling transform, vectorizer must learn vocabulary using fit or fit_transform.

2. Step 2: Identify error cause

   Code calls transform directly without fitting, so vocabulary is unknown, causing error.

3. Final Answer:

   fit_transform should be used before transform to learn vocabulary -> Option A

4. Quick Check:

   fit_transform first, then transform [OK]

Hint: Always fit vectorizer before transform [OK]

Common Mistakes:

• Calling transform without fit
• Assuming toarray() is invalid (it is valid)
• Thinking TfidfVectorizer can't handle multiple docs

5. You have two documents: doc1 = 'apple apple orange' and doc2 = 'apple banana banana'. Using TF-IDF, which word will have the highest weight in doc2 and why?

hard

A. 'banana', because it appears frequently in doc2 but less in doc1

B. 'apple', because it appears in both documents

C. 'orange', because it appears only in doc1

D. 'apple', because it appears most frequently overall

Solution

1. Step 1: Understand TF-IDF weighting

   TF-IDF gives higher weight to words frequent in one document but rare across all documents.

2. Step 2: Analyze word frequencies

   'banana' appears twice in doc2 but not in doc1, so it is rare overall but frequent in doc2.

3. Step 3: Compare with other words

   'apple' appears in both docs, so its IDF is lower. 'orange' is not in doc2, so no weight there.

4. Final Answer:

   'banana', because it appears frequently in doc2 but less in doc1 -> Option A

5. Quick Check:

   TF-IDF favors unique frequent words [OK]

Hint: TF-IDF boosts words frequent in one doc but rare overall [OK]

Common Mistakes:

• Choosing common words across docs as highest weight
• Confusing frequency with uniqueness
• Ignoring that words not in doc have zero weight