Target encoding in ML Python - Model Pipeline Trace

Model Pipeline - Target encoding

Target encoding replaces categories in a feature with the average value of the target for those categories. This helps models use categorical data better by turning categories into numbers that relate to the prediction goal.

Data Flow - 4 Stages

1Raw data input

1000 rows x 3 columns→Original dataset with categorical feature and target→1000 rows x 3 columns

Feature: Color = ['Red', 'Blue', 'Green', ...], Target: Sale = [1, 0, 1, ...]

↓

2Calculate target mean per category

1000 rows x 3 columns→Group by categorical feature and compute mean target→Number of unique categories x 2 columns

Color: Red -> mean target 0.7, Blue -> 0.4, Green -> 0.5

↓

3Replace categories with target mean

1000 rows x 3 columns→Map each category to its target mean value→1000 rows x 3 columns

Color: Red replaced by 0.7, Blue by 0.4, Green by 0.5

↓

4Model training

1000 rows x 3 columns→Train model using encoded feature and other features→Trained model

Model learns relationship between encoded color and sale

Training Trace - Epoch by Epoch

Loss
0.7 |****
0.6 |*** 
0.5 |**  
0.4 |*   
0.3 |    
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.6	Model starts learning, loss is high, accuracy low
2	0.5	0.72	Loss decreases, accuracy improves as model learns patterns
3	0.4	0.8	Model continues improving, better fit to data
4	0.35	0.85	Loss decreases further, accuracy rises
5	0.32	0.87	Model converges with good accuracy

Prediction Trace - 3 Layers

Layer 1: Input sample

Layer 2: Target encoding mapping

Layer 3: Model prediction

Model Quiz - 3 Questions

What does target encoding replace in the data?

A Target values with categories

B Categories with their average target value

C Categories with random numbers

D Missing values with zero

Key Insight

Target encoding helps convert categories into meaningful numbers based on the target. This makes it easier for models to find patterns and improve prediction accuracy.