Custom NER training basics in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of training loop snippet for custom NER

What will be the printed output after running this training loop snippet for 3 iterations?

NLP

import spacy
from spacy.training.example import Example

nlp = spacy.blank('en')
ner = nlp.add_pipe('ner')
ner.add_label('ANIMAL')

optimizer = nlp.begin_training()

TRAIN_DATA = [
    ('I have a dog', {'entities': [(7, 10, 'ANIMAL')]}),
    ('She owns a cat', {'entities': [(10, 13, 'ANIMAL')]})
]

for i in range(3):
    losses = {}
    for text, annotations in TRAIN_DATA:
        doc = nlp.make_doc(text)
        example = Example.from_dict(doc, annotations)
        nlp.update([example], sgd=optimizer, losses=losses)
    print(f'Iteration {i+1}, Losses: {losses}')

A

Iteration 1, Losses: {'ner': 0.5}
Iteration 2, Losses: {'ner': 0.3}
Iteration 3, Losses: {'ner': 0.1}

B

Iteration 1, Losses: {'ner': 0.0}
Iteration 2, Losses: {'ner': 0.0}
Iteration 3, Losses: {'ner': 0.0}

C

Iteration 1, Losses: {'ner': 0.5}
Iteration 2, Losses: {'ner': 0.0}
Iteration 3, Losses: {'ner': 0.0}

D

Iteration 1, Losses: {'ner': 0.0}
Iteration 2, Losses: {'ner': 0.3}
Iteration 3, Losses: {'ner': 0.1}

💡 Hint

Losses usually decrease as training progresses but start from a positive value.

The losses start positive and decrease as the model learns. They won't be zero initially because the model is blank and needs to learn the entity.

❓ Model Choice

intermediate

Choosing the right model architecture for custom NER

Which model architecture is best suited for training a custom Named Entity Recognition (NER) system from scratch?

A A convolutional neural network (CNN) designed for image classification

B A transformer-based model like BERT fine-tuned for token classification

C A recurrent neural network (RNN) with LSTM layers for sequence labeling

D A simple feedforward neural network without sequence context

💡 Hint

NER requires understanding context around each word in a sentence.

Transformer-based models like BERT capture context well and are state-of-the-art for NER tasks. CNNs for images or simple feedforward networks lack sequence understanding.

❓ Hyperparameter

advanced

Effect of batch size on custom NER training

What is the most likely effect of increasing the batch size during training of a custom NER model?

A Training becomes slower but model generalizes better

B Training becomes faster and model always achieves higher accuracy

C Training speed and model performance remain unchanged

D Training becomes faster but may lead to less stable updates

💡 Hint

Think about how many examples the model sees before updating weights.

Larger batch sizes process more data before updating weights, speeding training but sometimes causing less stable updates and poorer generalization.

❓ Metrics

advanced

Choosing the right metric for custom NER evaluation

Which metric best evaluates the performance of a custom NER model on a test set?

A Mean squared error between predicted and true entity labels

B Accuracy of token classification ignoring entity boundaries

C Precision, Recall, and F1-score based on exact entity matches

D Confusion matrix of sentence-level classification

💡 Hint

NER evaluation requires matching whole entities, not just tokens.

Precision, Recall, and F1-score computed on exact entity matches are standard for NER, capturing how well entities are found and classified.

🔧 Debug

expert

Identifying cause of poor entity recognition in custom NER

After training a custom NER model, it fails to recognize any entities in new sentences. Which is the most likely cause?

A The training data had no entity annotations or was empty

B The model was trained with too many epochs causing overfitting

C The optimizer was set to None during training

D The model was trained on a different language than the test sentences

💡 Hint

If the model never saw entities during training, it cannot learn to recognize them.

Without entity annotations in training data, the model never learns to detect entities, resulting in zero predictions.