NER with NLTK in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this NER chunking code?

Given the following Python code using NLTK for Named Entity Recognition (NER), what is the output?

NLP

import nltk
from nltk import word_tokenize, pos_tag, ne_chunk
sentence = "Apple is looking at buying U.K. startup for $1 billion"
tokens = word_tokenize(sentence)
pos_tags = pos_tag(tokens)
chunks = ne_chunk(pos_tags)
print(list(chunks))

A [('Apple', 'NNP'), ('is', 'VBZ'), ('looking', 'VBG'), ('at', 'IN'), Tree('ORGANIZATION', [('buying', 'VBG')]), ('U.K.', 'NNP'), ('startup', 'NN'), ('for', 'IN'), ('$', '$'), ('1', 'CD'), ('billion', 'CD')]

B [Tree('GPE', [('Apple', 'NNP')]), ('is', 'VBZ'), ('looking', 'VBG'), ('at', 'IN'), ('buying', 'VBG'), Tree('GPE', [('U.K.', 'NNP')]), ('startup', 'NN'), ('for', 'IN'), ('$', '$'), ('1', 'CD'), ('billion', 'CD')]

C [('Apple', 'NNP'), ('is', 'VBZ'), ('looking', 'VBG'), ('at', 'IN'), ('buying', 'VBG'), ('U.K.', 'NNP'), ('startup', 'NN'), ('for', 'IN'), ('$', '$'), ('1', 'CD'), ('billion', 'CD')]

D [Tree('ORGANIZATION', [('Apple', 'NNP')]), ('is', 'VBZ'), ('looking', 'VBG'), ('at', 'IN'), ('buying', 'VBG'), Tree('GPE', [('U.K.', 'NNP')]), ('startup', 'NN'), ('for', 'IN'), ('$', '$'), ('1', 'CD'), ('billion', 'CD')]

💡 Hint

Look at how NLTK labels named entities like organizations and geopolitical entities.

NLTK's ne_chunk labels 'Apple' as an ORGANIZATION and 'U.K.' as a GPE (Geopolitical Entity). The output is a list mixing Tree objects for entities and tuples for other tokens.

🧠 Conceptual

intermediate

Which NLTK function performs POS tagging before NER?

In the NLTK pipeline for Named Entity Recognition, which function is responsible for assigning part-of-speech tags to tokens before chunking?

A ne_chunk()

B word_tokenize()

C pos_tag()

D sent_tokenize()

💡 Hint

POS tags describe the role of each word like noun or verb.

pos_tag() assigns part-of-speech tags to tokens, which ne_chunk() then uses to identify named entities.

❓ Hyperparameter

advanced

What is the effect of using binary=True in ne_chunk()?

In NLTK's ne_chunk function, setting binary=True changes the output. What does this parameter do?

A It merges all named entities into a single 'NE' label without specific types.

B It disables named entity recognition and returns only POS tags.

C It enables recognition of only person names, ignoring other entity types.

D It outputs entities as plain strings instead of Tree objects.

💡 Hint

Think about simplifying entity categories.

binary=True groups all named entities under a generic 'NE' label, ignoring specific types like ORGANIZATION or GPE.

🔧 Debug

advanced

Why does this NER code raise a LookupError?

Consider this code snippet: import nltk sentence = "Google is a tech giant" tokens = nltk.word_tokenize(sentence) pos_tags = nltk.pos_tag(tokens) chunks = nltk.ne_chunk(pos_tags) When running, it raises a LookupError about missing 'maxent_ne_chunker'. What is the cause?

A The sentence contains unknown words causing the error.

B The 'maxent_ne_chunker' model is not downloaded in NLTK data.

C pos_tag() was not called before ne_chunk().

D word_tokenize() requires a language parameter that is missing.

💡 Hint

Check if all required NLTK models are installed.

NLTK requires the 'maxent_ne_chunker' model to perform named entity chunking. If not downloaded, it raises LookupError.

❓ Model Choice

expert

Which NLTK model is used internally by ne_chunk for NER?

NLTK's ne_chunk function uses a pre-trained model internally. Which model does it use for Named Entity Recognition?

A A Maximum Entropy classifier trained on the ACE corpus

B A Hidden Markov Model trained on the CoNLL 2003 dataset

C A Maximum Entropy classifier trained on the CONLL 2000 corpus

D A Maximum Entropy classifier trained on the ACE corpus and Treebank data

💡 Hint

Consider the training data NLTK mentions for ne_chunk.

NLTK's ne_chunk uses a Maximum Entropy classifier trained on the ACE corpus for NER.