Evaluation metrics (accuracy, F1, confusion matrix) in NLP - Model Pipeline Trace

Model Pipeline - Evaluation metrics (accuracy, F1, confusion matrix)

This pipeline shows how a text classification model is evaluated using accuracy, F1 score, and confusion matrix. These metrics help us understand how well the model predicts categories.

Data Flow - 4 Stages

1Raw Text Data

1000 rows x 1 column→Collect sentences with labels (e.g., positive/negative)→1000 rows x 2 columns

Sentence: 'I love this movie', Label: positive

↓

2Text Preprocessing

1000 rows x 2 columns→Clean text, tokenize, convert to vectors→1000 rows x 100 features

Vector for 'I love this movie' with 100 numbers

↓

3Model Prediction

1000 rows x 100 features→Model predicts labels for each input→1000 rows x 1 column

Predicted label: positive

↓

4Evaluation Metrics Calculation

1000 rows x 2 columns (true and predicted labels)→Calculate accuracy, F1 score, confusion matrix→Summary metrics and matrix

Accuracy: 0.92, F1: 0.93, Confusion matrix: [[420, 30], [50, 500]]

Training Trace - Epoch by Epoch

Loss
0.7 | *
0.6 | **
0.5 | ***
0.4 | ****
0.3 | *****
    +------------
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.6	Model starts learning, accuracy is low
2	0.5	0.72	Loss decreases, accuracy improves
3	0.4	0.78	Model is learning patterns better
4	0.32	0.83	Good improvement in accuracy
5	0.28	0.85	Model converges with stable accuracy

Prediction Trace - 5 Layers

Layer 1: Input Text

Layer 2: Text Vectorization

Layer 3: Model Prediction

Layer 4: Class Decision

Layer 5: Evaluation Metrics

Model Quiz - 3 Questions

What does accuracy measure in this model?

A The number of features used in the model

B The average length of input sentences

C The percentage of correct predictions out of all predictions

D The time taken to train the model

Key Insight

Accuracy alone can be misleading if classes are imbalanced. Using F1 score and confusion matrix together gives a clearer picture of model performance, especially in text classification.