Bag of Words (CountVectorizer) in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of CountVectorizer on simple text

What is the output of the following code snippet using CountVectorizer from scikit-learn?

NLP

from sklearn.feature_extraction.text import CountVectorizer
corpus = ['apple banana apple', 'banana orange']
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(corpus)
result = X.toarray()
vocab = vectorizer.get_feature_names_out()
print(vocab)
print(result)

A ['apple' 'orange' 'banana']\n[[2 0 1]\n [0 1 1]]

B ['banana' 'apple' 'orange']\n[[1 2 0]\n [1 0 1]]

C ['apple' 'banana' 'orange']\n[[2 1 0]\n [0 1 1]]

D ['apple' 'banana' 'orange']\n[[1 2 0]\n [0 1 1]]

💡 Hint

CountVectorizer sorts vocabulary alphabetically and counts word occurrences per document.

The CountVectorizer creates a vocabulary sorted alphabetically: ['apple', 'banana', 'orange']. It counts how many times each word appears in each document. The first document has 'apple' twice and 'banana' once, no 'orange'. The second document has 'banana' once and 'orange' once.

🧠 Conceptual

intermediate

Understanding vocabulary size in CountVectorizer

Given the corpus: ['cat dog', 'dog mouse', 'cat mouse dog'], what is the vocabulary size created by CountVectorizer with default settings?

A 3

B 5

C 4

D 2

💡 Hint

CountVectorizer creates one vocabulary word per unique token across all documents.

The unique words are 'cat', 'dog', and 'mouse'. So the vocabulary size is 3.

❓ Hyperparameter

advanced

Effect of stop_words parameter in CountVectorizer

What will be the vocabulary output of CountVectorizer when applied to ['the cat sat', 'the dog barked'] with stop_words='english'?

A ['the', 'cat', 'sat', 'dog', 'barked']

B ['barked', 'cat', 'dog', 'sat']

C ['cat', 'dog']

D ['the']

💡 Hint

The stop_words='english' removes common English words like 'the'.

Stop words like 'the' are removed. Remaining words are 'cat', 'sat', 'dog', 'barked'. Vocabulary is sorted alphabetically.

❓ Metrics

advanced

Calculating document frequency with CountVectorizer

Using CountVectorizer on ['apple apple banana', 'banana orange', 'apple orange orange'], what is the document frequency (number of documents containing the word) for 'apple'?

A 2

B 3

C 1

D 0

💡 Hint

Document frequency counts in how many documents the word appears at least once.

'apple' appears in the first and third documents, so document frequency is 2.

🔧 Debug

expert

Identifying error in CountVectorizer usage

What error will the following code raise? from sklearn.feature_extraction.text import CountVectorizer corpus = ['hello world', 123, 'hello'] vectorizer = CountVectorizer() X = vectorizer.fit_transform(corpus)

A AttributeError: 'int' object has no attribute 'lower'

B ValueError: empty vocabulary; perhaps the documents only contain stop words

C No error, code runs successfully

D TypeError: expected string or bytes-like object

💡 Hint

CountVectorizer expects all documents to be strings.

The corpus contains an integer 123, which causes CountVectorizer to raise a TypeError because it expects strings.