Label smoothing in PyTorch - Model Pipeline Trace

Model Pipeline - Label smoothing

This pipeline shows how label smoothing helps a classification model learn better by softening the target labels. Instead of hard 0 or 1 labels, it uses slightly softened labels to reduce overconfidence and improve generalization.

Data Flow - 4 Stages

1Raw data input

1000 rows x 10 columns→Load dataset with 10 features per sample→1000 rows x 10 columns

[5.1, 3.5, 1.4, ..., 0.2]

↓

2Train/test split

1000 rows x 10 columns→Split data into training (80%) and testing (20%) sets→800 rows x 10 columns (train), 200 rows x 10 columns (test)

[5.1, 3.5, 1.4, ..., 0.2]

↓

3Label smoothing applied

800 rows x 3 classes (one-hot labels)→Convert hard labels (0 or 1) to soft labels (e.g., 0.9 and 0.05)→800 rows x 3 classes (smoothed labels)

[0.05, 0.9, 0.05]

↓

4Model training

800 rows x 10 features→Train neural network with smoothed labels→Trained model

Neural network weights updated

Training Trace - Epoch by Epoch

Loss
1.2 |*       
1.0 | **     
0.8 |  ***   
0.6 |   **** 
0.4 |        
    +--------
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.20	0.55	Loss starts high, accuracy moderate
2	0.95	0.65	Loss decreases, accuracy improves
3	0.80	0.72	Model learns better with smoothed labels
4	0.70	0.78	Loss continues to decrease steadily
5	0.62	0.82	Accuracy improves, model generalizes well

Prediction Trace - 4 Layers

Layer 1: Input layer

Layer 2: Hidden layers with ReLU

Layer 3: Output layer with softmax

Layer 4: Compare with smoothed label

Model Quiz - 3 Questions

What is the main purpose of label smoothing in training?

A To increase the number of training samples

B To speed up the training process

C To prevent the model from becoming too confident

D To reduce the number of features

Key Insight

Label smoothing helps the model avoid being too sure about its predictions. This leads to better learning and improved accuracy by making the model more flexible and less prone to mistakes on new data.