Custom NER training basics in NLP - Model Pipeline Trace

Model Pipeline - Custom NER training basics

This pipeline trains a model to recognize custom named entities in text, like names or places. It starts with raw text, adds labels, trains a model, and then checks how well it learned.

Data Flow - 5 Stages

1Raw Text Input

1000 sentences→Collect sentences with entities to label→1000 sentences

"Apple is looking at buying U.K. startup for $1 billion."

↓

2Annotation

1000 sentences→Label entities in sentences (e.g., 'Apple' as ORG)→1000 sentences with entity labels

"Apple" labeled as ORG, "U.K." labeled as GPE

↓

3Data Preprocessing

1000 sentences with labels→Convert text and labels into token-level format→1000 token sequences with entity tags

[('Apple', 'ORG'), ('is', 'O'), ('looking', 'O'), ...]

↓

4Model Training

1000 token sequences with tags→Train NER model to predict entity tags→Trained NER model

Model learns to tag 'Apple' as ORG

↓

5Evaluation

Validation sentences with labels→Calculate accuracy and F1 score→Performance metrics

Accuracy: 0.92, F1 score: 0.89

Training Trace - Epoch by Epoch

Loss
1.0 |****
0.8 |*** 
0.6 |**  
0.4 |*   
0.2 |    
    +-----
     1 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.60	Model starts learning entity patterns
2	0.60	0.75	Loss decreases, accuracy improves
3	0.45	0.82	Model better at recognizing entities
4	0.35	0.87	Training converging, accuracy rising
5	0.28	0.90	Good balance of loss and accuracy

Prediction Trace - 4 Layers

Layer 1: Tokenization

Layer 2: Feature Extraction

Layer 3: NER Model Prediction

Layer 4: Post-processing

Model Quiz - 3 Questions

What happens during the annotation stage?

A Splitting sentences into tokens

B Training the model

C Labeling entities in sentences

D Calculating accuracy

Key Insight

Training a custom NER model involves labeling text with entities, converting text into tokens, and teaching the model to recognize patterns. As training progresses, loss decreases and accuracy improves, showing the model learns to identify entities better.

Practice

1. What is the main goal of custom NER training in NLP?

easy

A. To summarize long documents automatically

B. To teach the model to recognize specific words or phrases you label

C. To translate text from one language to another

D. To generate new text based on a prompt

Solution

1. Step 1: Understand what NER means

   NER stands for Named Entity Recognition, which means finding specific words or phrases in text.

2. Step 2: Identify the purpose of custom training

   Custom NER training teaches the model to find your special labeled words, not general tasks like translation or summarization.

3. Final Answer:

   To teach the model to recognize specific words or phrases you label -> Option B

4. Quick Check:

   Custom NER = Recognize labeled words [OK]

Hint: Custom NER means teaching model your special words [OK]

Common Mistakes:

• Confusing NER with translation or summarization
• Thinking NER generates new text
• Assuming NER works without labeled data

2. Which of the following is the correct way to label a sentence for custom NER training in Python spaCy format?

easy

A. ('Apple is a company', {'entities': [(0, 5, 'ORG')]})

B. ('Apple is a company', {'labels': [(0, 5, 'ORG')]})

C. ('Apple is a company', {'entities': [(6, 7, 'ORG')]})

D. ('Apple is a company', {'entities': [(0, 5, 'PERSON')]})

Solution

1. Step 1: Check the labeling key

   spaCy uses the 'entities' key, not 'labels', to hold labeled spans.

2. Step 2: Verify the span and label

   Span (0,5) covers 'Apple' correctly, and label 'ORG' (organization) fits. A span like (6,7,'ORG') points to the wrong position, and 'PERSON' is incorrect for a company.

3. Final Answer:

   ('Apple is a company', {'entities': [(0, 5, 'ORG')]}) -> Option A

4. Quick Check:

   Correct key and span = ('Apple is a company', {'entities': [(0, 5, 'ORG')]}) [OK]

Hint: Use 'entities' key with correct span and label [OK]

Common Mistakes:

• Using 'labels' instead of 'entities'
• Incorrect character span for entity
• Wrong entity type label

3. Given this training data snippet for custom NER:

TRAIN_DATA = [
  ('I love Paris', {'entities': [(7, 12, 'GPE')]})
]

What will the model predict for the sentence 'I love Paris' after training?

medium

A. [] (no entities)

B. [('I', 'GPE')]

C. [('Paris', 'GPE')]

D. [('love', 'GPE')]

Solution

1. Step 1: Understand the labeled entity

   The training data labels 'Paris' from character 7 to 12 as 'GPE' (Geopolitical entity).

2. Step 2: Predict model output after training

   The model learns to recognize 'Paris' as 'GPE' and should predict [('Paris', 'GPE')] for the same sentence.

3. Final Answer:

   [('Paris', 'GPE')] -> Option C

4. Quick Check:

   Entity span matches 'Paris' = [('Paris', 'GPE')] [OK]

Hint: Model predicts labeled spans from training data [OK]

Common Mistakes:

• Confusing entity span with other words
• Expecting no entities if training is done
• Mixing entity labels

4. You wrote this code to add a new entity label to your NER model:

ner.add_label('ANIMAL')

But after training, the model never detects 'ANIMAL' entities. What is the most likely mistake?

medium

A. The label 'ANIMAL' is reserved and cannot be used

B. You used the wrong method name; it should be add_entity()

C. You need to call ner.remove_label('ANIMAL') before adding

D. You forgot to include training examples with 'ANIMAL' labels

Solution

1. Step 1: Check the method usage

   ner.add_label('ANIMAL') is correct to add a new label. There is no add_entity() method, no need to call remove_label first, and 'ANIMAL' is not reserved.

2. Step 2: Verify training data

   Model learns from examples. Without training examples labeled 'ANIMAL', model cannot detect it.

3. Final Answer:

   You forgot to include training examples with 'ANIMAL' labels -> Option D

4. Quick Check:

   Training data needed for new labels = You forgot to include training examples with 'ANIMAL' labels [OK]

Hint: Add labeled examples for new entity labels [OK]

Common Mistakes:

• Assuming adding label alone trains model
• Using wrong method names
• Thinking labels are reserved keywords

5. You want to train a custom NER model to recognize two new entity types: 'FOOD' and 'DRINK'. You have labeled training data for both. Which of the following is the best approach to ensure the model learns both correctly?

hard

A. Add both labels with ner.add_label(), include balanced training examples for each, and train in multiple iterations

B. Add only 'FOOD' label first, train fully, then add 'DRINK' label and train again

C. Train the model without adding labels explicitly; it will learn automatically

D. Add labels but use only examples for 'FOOD' to avoid confusion

Solution

1. Step 1: Add all new labels before training

   Adding both 'FOOD' and 'DRINK' labels upfront ensures model knows what to learn.

2. Step 2: Provide balanced training data and train iteratively

   Balanced examples for both labels and multiple training loops help model learn both well.

3. Final Answer:

   Add both labels with ner.add_label(), include balanced training examples for each, and train in multiple iterations -> Option A

4. Quick Check:

   All labels + balanced data + training = Add both labels with ner.add_label(), include balanced training examples for each, and train in multiple iterations [OK]

Hint: Add all labels and balanced data before training [OK]

Common Mistakes:

• Adding labels one by one with separate training
• Skipping label addition
• Training with unbalanced or missing examples