Custom NER training basics in NLP - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does NER stand for in NLP?

NER stands for Named Entity Recognition. It is a process to find and classify names, places, dates, and other important information in text.

beginner

Why do we train a custom NER model?

We train a custom NER model to recognize specific types of entities that are important for our task but not covered by general models, like product names or medical terms.

beginner

What is the role of annotated data in custom NER training?

Annotated data is text where entities are marked with their correct labels. This data teaches the model what to look for during training.

intermediate

Name one common metric used to evaluate NER models.

F1 score is commonly used. It balances precision (correctness) and recall (completeness) to measure model performance.

beginner

What is a typical step after training a custom NER model?

After training, you test the model on new text to see how well it finds entities, then improve it by adding more data or tuning settings.

What is the main purpose of Named Entity Recognition?

A To translate text from one language to another

B To generate new text automatically

C To summarize long documents

D To find and label important words like names and places in text

NER identifies and labels entities such as names, places, and dates in text.

Why might you need to train a custom NER model instead of using a pre-trained one?

A To reduce the size of the model

B Because pre-trained models are always inaccurate

C To recognize special entity types not in general models

D To translate text faster

Custom training helps the model learn to find entities specific to your needs.

What is annotated data in the context of NER training?

A Text translated into another language

B Text with entities marked and labeled

C Text summarized into short sentences

D Text generated by the model

Annotated data shows the model examples of correct entity labels.

Which metric combines precision and recall to evaluate NER models?

A F1 score

B Accuracy

C Loss

D BLEU score

F1 score balances precision and recall for a fair evaluation.

What is a common next step after training a custom NER model?

A Test it on new text and improve if needed

B Delete the training data

C Use it to translate languages

D Convert it into a chatbot

Testing helps check how well the model works and guides improvements.

Explain the process and importance of annotating data for custom NER training.

Think about how the model knows what to find in text.

Describe how you would evaluate a custom NER model after training.

Consider how to check if the model is good at finding entities.

Practice

1. What is the main goal of custom NER training in NLP?

easy

A. To summarize long documents automatically

B. To teach the model to recognize specific words or phrases you label

C. To translate text from one language to another

D. To generate new text based on a prompt

Solution

1. Step 1: Understand what NER means

   NER stands for Named Entity Recognition, which means finding specific words or phrases in text.

2. Step 2: Identify the purpose of custom training

   Custom NER training teaches the model to find your special labeled words, not general tasks like translation or summarization.

3. Final Answer:

   To teach the model to recognize specific words or phrases you label -> Option B

4. Quick Check:

   Custom NER = Recognize labeled words [OK]

Hint: Custom NER means teaching model your special words [OK]

Common Mistakes:

• Confusing NER with translation or summarization
• Thinking NER generates new text
• Assuming NER works without labeled data

2. Which of the following is the correct way to label a sentence for custom NER training in Python spaCy format?

easy

A. ('Apple is a company', {'entities': [(0, 5, 'ORG')]})

B. ('Apple is a company', {'labels': [(0, 5, 'ORG')]})

C. ('Apple is a company', {'entities': [(6, 7, 'ORG')]})

D. ('Apple is a company', {'entities': [(0, 5, 'PERSON')]})

Solution

1. Step 1: Check the labeling key

   spaCy uses the 'entities' key, not 'labels', to hold labeled spans.

2. Step 2: Verify the span and label

   Span (0,5) covers 'Apple' correctly, and label 'ORG' (organization) fits. A span like (6,7,'ORG') points to the wrong position, and 'PERSON' is incorrect for a company.

3. Final Answer:

   ('Apple is a company', {'entities': [(0, 5, 'ORG')]}) -> Option A

4. Quick Check:

   Correct key and span = ('Apple is a company', {'entities': [(0, 5, 'ORG')]}) [OK]

Hint: Use 'entities' key with correct span and label [OK]

Common Mistakes:

• Using 'labels' instead of 'entities'
• Incorrect character span for entity
• Wrong entity type label

3. Given this training data snippet for custom NER:

TRAIN_DATA = [
  ('I love Paris', {'entities': [(7, 12, 'GPE')]})
]

What will the model predict for the sentence 'I love Paris' after training?

medium

A. [] (no entities)

B. [('I', 'GPE')]

C. [('Paris', 'GPE')]

D. [('love', 'GPE')]

Solution

1. Step 1: Understand the labeled entity

   The training data labels 'Paris' from character 7 to 12 as 'GPE' (Geopolitical entity).

2. Step 2: Predict model output after training

   The model learns to recognize 'Paris' as 'GPE' and should predict [('Paris', 'GPE')] for the same sentence.

3. Final Answer:

   [('Paris', 'GPE')] -> Option C

4. Quick Check:

   Entity span matches 'Paris' = [('Paris', 'GPE')] [OK]

Hint: Model predicts labeled spans from training data [OK]

Common Mistakes:

• Confusing entity span with other words
• Expecting no entities if training is done
• Mixing entity labels

4. You wrote this code to add a new entity label to your NER model:

ner.add_label('ANIMAL')

But after training, the model never detects 'ANIMAL' entities. What is the most likely mistake?

medium

A. The label 'ANIMAL' is reserved and cannot be used

B. You used the wrong method name; it should be add_entity()

C. You need to call ner.remove_label('ANIMAL') before adding

D. You forgot to include training examples with 'ANIMAL' labels

Solution

1. Step 1: Check the method usage

   ner.add_label('ANIMAL') is correct to add a new label. There is no add_entity() method, no need to call remove_label first, and 'ANIMAL' is not reserved.

2. Step 2: Verify training data

   Model learns from examples. Without training examples labeled 'ANIMAL', model cannot detect it.

3. Final Answer:

   You forgot to include training examples with 'ANIMAL' labels -> Option D

4. Quick Check:

   Training data needed for new labels = You forgot to include training examples with 'ANIMAL' labels [OK]

Hint: Add labeled examples for new entity labels [OK]

Common Mistakes:

• Assuming adding label alone trains model
• Using wrong method names
• Thinking labels are reserved keywords

5. You want to train a custom NER model to recognize two new entity types: 'FOOD' and 'DRINK'. You have labeled training data for both. Which of the following is the best approach to ensure the model learns both correctly?

hard

A. Add both labels with ner.add_label(), include balanced training examples for each, and train in multiple iterations

B. Add only 'FOOD' label first, train fully, then add 'DRINK' label and train again

C. Train the model without adding labels explicitly; it will learn automatically

D. Add labels but use only examples for 'FOOD' to avoid confusion

Solution

1. Step 1: Add all new labels before training

   Adding both 'FOOD' and 'DRINK' labels upfront ensures model knows what to learn.

2. Step 2: Provide balanced training data and train iteratively

   Balanced examples for both labels and multiple training loops help model learn both well.

3. Final Answer:

   Add both labels with ner.add_label(), include balanced training examples for each, and train in multiple iterations -> Option A

4. Quick Check:

   All labels + balanced data + training = Add both labels with ner.add_label(), include balanced training examples for each, and train in multiple iterations [OK]

Hint: Add all labels and balanced data before training [OK]

Common Mistakes:

• Adding labels one by one with separate training
• Skipping label addition
• Training with unbalanced or missing examples