Lemmatization in spaCy in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this spaCy lemmatization code?

Given the following code snippet using spaCy, what will be the printed list of lemmas?

NLP

import spacy
nlp = spacy.load('en_core_web_sm')
doc = nlp('The striped bats are hanging on their feet for best')
lemmas = [token.lemma_ for token in doc]
print(lemmas)

A ['the', 'striped', 'bat', 'be', 'hang', 'on', 'their', 'foot', 'for', 'good']

B ['the', 'striped', 'bat', 'be', 'hang', 'on', 'their', 'feet', 'for', 'best']

C ['The', 'striped', 'bats', 'are', 'hanging', 'on', 'their', 'feet', 'for', 'best']

D ['the', 'striped', 'bat', 'are', 'hang', 'on', 'their', 'feet', 'for', 'best']

💡 Hint

Look at how spaCy converts plural nouns and verbs to their base forms.

spaCy lemmatizes 'bats' to 'bat', 'are' to 'be', 'hanging' to 'hang', and keeps 'feet' as 'feet' because it's irregular plural. 'best' remains unchanged as it's an adjective.

❓ Model Choice

intermediate

Which spaCy model is best for accurate lemmatization?

You want to perform lemmatization on English text with good accuracy and speed. Which spaCy model should you choose?

A en_core_web_sm (small model)

B en_vectors_web_lg (only word vectors, no lemmatization)

C en_core_web_lg (large model)

D en_core_web_md (medium model)

💡 Hint

Larger models usually have better linguistic features for tasks like lemmatization.

The large model (en_core_web_lg) includes more detailed linguistic data and performs better lemmatization than smaller models. The vectors-only model does not provide lemmatization.

❓ Metrics

advanced

Which metric best evaluates lemmatization quality?

You have a dataset with gold-standard lemmas and your spaCy model's predicted lemmas. Which metric best measures lemmatization accuracy?

A Exact match accuracy

B Recall

C Precision

D F1 score

💡 Hint

Lemmatization is about exact word form matches.

Exact match accuracy counts how many predicted lemmas exactly match the gold lemmas, which is the most direct measure for lemmatization quality.

🔧 Debug

advanced

Why does this spaCy lemmatization code raise an error?

Consider this code snippet: import spacy nlp = spacy.load('en_core_web_sm') text = 'Cats running fast' doc = nlp(text) lemmas = [token.lemma for token in doc] print(lemmas) Why does it raise an AttributeError?

A 'doc' object is not iterable error

B 'nlp' object is not callable error due to missing parentheses

C 'text' variable is not defined before use

D 'Token' object has no attribute 'lemma' because the correct attribute is 'lemma_'

💡 Hint

Check the attribute name for token lemmas in spaCy.

spaCy tokens have the attribute 'lemma_' (with underscore) for the lemma string. Using 'lemma' without underscore causes AttributeError.

🧠 Conceptual

expert

Why might spaCy lemmatization keep 'feet' as 'feet' instead of 'foot'?

In spaCy, the word 'feet' is lemmatized as 'feet' instead of the expected singular 'foot'. What is the most likely reason?

A spaCy's lemmatizer uses a dictionary-based approach that sometimes keeps irregular plurals unchanged

B The lemmatizer relies on part-of-speech tags and 'feet' is tagged as plural noun but lemmatizer lacks irregular plural rules

C The model's vocabulary does not include 'foot' so it cannot lemmatize 'feet' correctly

D spaCy treats 'feet' as a plural noun but does not normalize irregular plurals to singular

💡 Hint

Think about how dictionary-based lemmatizers handle irregular forms.

spaCy's lemmatizer uses a dictionary and rules. Sometimes irregular plurals like 'feet' are kept unchanged if the dictionary entry maps 'feet' to 'feet' instead of 'foot'.