NLPml~20 mins

Entity types (PERSON, ORG, LOC, DATE) in NLP - ML Experiment: Train & Evaluate

Choose your learning style10 modes available

Learn Why Deep Model Try Challenge Experiment Recall Metrics

Start learning this pattern below

Jump into concepts and practice - no test required

Recommended

Test this pattern10 questions across easy, medium, and hard to know if this pattern is strong

Experiment - Entity types (PERSON, ORG, LOC, DATE)

Problem:You want to build a model that can recognize named entities in text, specifically people (PERSON), organizations (ORG), locations (LOC), and dates (DATE). The current model identifies entities but often confuses entity types or misses some entities.

Current Metrics:Training accuracy: 92%, Validation accuracy: 75%, Validation F1-score: 0.70

Issue:The model is overfitting: training accuracy is high but validation accuracy and F1-score are much lower, indicating poor generalization.

Your Task

Reduce overfitting to improve validation accuracy to at least 85% and validation F1-score to at least 0.80, while keeping training accuracy below 90%.

You can only adjust model architecture and training hyperparameters.

You cannot change the dataset or add more data.

You must keep the entity types limited to PERSON, ORG, LOC, and DATE.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

NLP

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, Bidirectional, LSTM, Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Assume X_train, y_train, X_val, y_val are preprocessed and ready

vocab_size = 10000  # example vocabulary size
embedding_dim = 64
max_len = 100  # max length of input sequences
num_classes = 5  # 4 entity types + 1 for 'O' (no entity)

model = Sequential([
    Embedding(vocab_size, embedding_dim, input_length=max_len),
    Bidirectional(LSTM(64, return_sequences=True)),
    Dropout(0.5),
    Dense(64, activation='relu'),
    Dropout(0.5),
    Dense(num_classes, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0005),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)

history = model.fit(X_train, y_train,
                    epochs=20,
                    batch_size=32,
                    validation_data=(X_val, y_val),
                    callbacks=[early_stop])

Added dropout layers after LSTM and Dense layers to reduce overfitting.

Lowered learning rate to 0.0005 for smoother convergence.

Added early stopping to stop training when validation loss stops improving.

Results Interpretation

Before: Training accuracy: 92%, Validation accuracy: 75%, Validation F1-score: 0.70

After: Training accuracy: 88%, Validation accuracy: 86%, Validation F1-score: 0.82

Adding dropout and early stopping helped reduce overfitting, improving validation accuracy and F1-score while slightly lowering training accuracy. This shows how controlling model complexity and training duration helps models generalize better.

Bonus Experiment

Try using a pretrained language model like BERT for named entity recognition on the same dataset.

💡 Hint

Use a pretrained transformer model with a token classification head and fine-tune it on your entity dataset for potentially better accuracy.

Practice

(1/5)

1. Which entity type label would you use to mark the name "Albert Einstein" in a text?

easy

A. PERSON

B. ORG

C. LOC

D. DATE

Entity types (PERSON, ORG, LOC, DATE) in NLP - ML Experiment: Train & Evaluate

Start learning this pattern below

Practice

Solution

Step 1: Understand entity types

Step 2: Match the example to entity type

Final Answer:

Quick Check:

Solution

Step 1: Identify what Google represents

Step 2: Match to entity type

Final Answer:

Quick Check:

Solution

Step 1: Identify each entity type

Step 2: Match entities to types in order

Final Answer:

Quick Check:

Solution

Step 1: Understand the entity "Amazon"

Step 2: Correct entity type for Amazon

Final Answer:

Quick Check:

Solution

Step 1: Identify entities to extract

Step 2: Match entity types for locations and dates

Final Answer:

Quick Check: