Feature union in ML Python - Model Pipeline Trace

Model Pipeline - Feature union

This pipeline combines different sets of features from the same data to help the model learn better. It joins features side-by-side so the model sees more information at once.

Data Flow - 5 Stages

1Raw data input

1000 rows x 5 columns→Collect original features→1000 rows x 5 columns

[[5.1, 3.5, 1.4, 0.2, 0], [4.9, 3.0, 1.4, 0.2, 0], ...]

↓

2Feature extraction A

1000 rows x 5 columns→Extract numeric features (e.g., mean, max)→1000 rows x 3 columns

[[3.05, 5.1, 1.4], [2.7, 4.9, 1.4], ...]

↓

3Feature extraction B

1000 rows x 5 columns→Extract categorical features (e.g., one-hot encoding)→1000 rows x 4 columns

[[1, 0, 0, 0], [1, 0, 0, 0], ...]

↓

4Feature union

Two sets: 1000 rows x 3 columns and 1000 rows x 4 columns→Combine features side-by-side→1000 rows x 7 columns

[[3.05, 5.1, 1.4, 1, 0, 0, 0], [2.7, 4.9, 1.4, 1, 0, 0, 0], ...]

↓

5Model training

1000 rows x 7 columns→Train classifier on combined features→Trained model

Model learns to predict target labels

Training Trace - Epoch by Epoch

Loss
0.7 |****
0.6 |*** 
0.5 |**  
0.4 |*   
0.3 |    
0.2 |    
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.60	Model starts learning with moderate loss and accuracy
2	0.48	0.75	Loss decreases and accuracy improves as model learns
3	0.35	0.85	Model shows good learning progress
4	0.28	0.90	Loss continues to decrease, accuracy rises
5	0.22	0.93	Model converges with low loss and high accuracy

Prediction Trace - 5 Layers

Layer 1: Input sample

Layer 2: Feature extraction A

Layer 3: Feature extraction B

Layer 4: Feature union

Layer 5: Model prediction

Model Quiz - 3 Questions

What does the feature union step do in the pipeline?

A Removes duplicate features

B Combines different feature sets side-by-side

C Splits data into training and test sets

D Normalizes all features to the same scale

Key Insight

Feature union helps the model by combining different types of features into one set, giving it more information to learn from. This usually improves accuracy and helps the model understand the data better.