Time series evaluation metrics in ML Python - Model Pipeline Trace

Model Pipeline - Time series evaluation metrics

This pipeline shows how time series data is prepared, a forecasting model is trained, and then evaluated using common time series metrics like MAE, MSE, and RMSE to measure prediction accuracy.

Data Flow - 6 Stages

1Raw time series data

1000 time steps x 1 feature→Collect sequential data points over time→1000 time steps x 1 feature

Daily temperature readings for 1000 days

↓

2Train-test split

1000 time steps x 1 feature→Split data into training (80%) and testing (20%) sets→Train: 800 time steps x 1 feature, Test: 200 time steps x 1 feature

First 800 days for training, last 200 days for testing

↓

3Feature scaling

Train: 800 x 1, Test: 200 x 1→Normalize values to range 0-1 using Min-Max scaling→Train: 800 x 1 scaled, Test: 200 x 1 scaled

Temperature scaled so min=0 and max=1 in training set

↓

4Model training

Train: 800 x 1 scaled→Train forecasting model (e.g., LSTM) on training data→Trained model

Model learns to predict next day temperature from past days

↓

5Prediction on test set

Test: 200 x 1 scaled→Model predicts temperature values for test time steps→Predictions: 200 x 1 scaled

Predicted temperatures for days 801 to 1000

↓

6Evaluation metrics calculation

Predictions: 200 x 1, Actual test values: 200 x 1→Calculate MAE, MSE, RMSE between predicted and actual values→Scalar metric values

MAE=0.05, MSE=0.004, RMSE=0.063

Training Trace - Epoch by Epoch

Loss
0.12 |*       
0.10 | *      
0.08 |  *     
0.06 |   *    
0.04 |    **  
0.02 |      * 
     +--------
      1 5 10 20 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.12	N/A	Initial loss is high as model starts learning
5	0.08	N/A	Loss decreases steadily, model improving
10	0.05	N/A	Loss lower, model predictions getting closer to actual
15	0.04	N/A	Loss stabilizes, model converging
20	0.035	N/A	Final epoch with lowest loss, good fit achieved

Prediction Trace - 6 Layers

Layer 1: Input scaled test data

Layer 2: Model prediction

Layer 3: Inverse scaling

Layer 4: Calculate MAE

Layer 5: Calculate MSE

Layer 6: Calculate RMSE

Model Quiz - 3 Questions

Which metric measures the average absolute difference between predicted and actual values?

A RMSE

B MSE

C MAE

D Accuracy

Key Insight

Time series evaluation metrics like MAE, MSE, and RMSE help us understand how close our model's predictions are to actual values. Watching loss decrease during training shows the model is learning patterns in the data. Using these metrics together gives a fuller picture of model accuracy and error types.