Time series evaluation metrics in ML Python

Introduction

We use time series evaluation metrics to check how well our model predicts future values based on past data. This helps us know if our model is good or needs improvement.

When forecasting daily sales for a store to plan inventory.

When predicting temperature changes for weather forecasts.

When estimating stock prices for investment decisions.

When monitoring energy consumption to optimize usage.

When analyzing website traffic to prepare for high demand.

Syntax

ML Python

from sklearn.metrics import mean_absolute_error, mean_squared_error
import numpy as np

# y_true: actual values
# y_pred: predicted values

mae = mean_absolute_error(y_true, y_pred)
mse = mean_squared_error(y_true, y_pred)
rmse = np.sqrt(mse)

mean_absolute_error (MAE) measures average absolute difference between actual and predicted values.

mean_squared_error (MSE) squares the differences before averaging, penalizing bigger errors more.

root mean squared error (RMSE) is the square root of MSE, giving error in original units.

Examples

Calculates MAE for small lists of actual and predicted values.

ML Python

mae = mean_absolute_error([3, 5, 2], [2.5, 5, 2.1])
print(mae)

Calculates MSE for the same values, showing squared error average.

ML Python

mse = mean_squared_error([3, 5, 2], [2.5, 5, 2.1])
print(mse)

Calculates RMSE, which is easier to interpret as it is in the same scale as the data.

ML Python

rmse = np.sqrt(mean_squared_error([3, 5, 2], [2.5, 5, 2.1]))
print(rmse)

Sample Model

This program compares actual and predicted time series values using MAE, MSE, and RMSE to evaluate prediction accuracy.

ML Python

from sklearn.metrics import mean_absolute_error, mean_squared_error
import numpy as np

# Actual values of a time series
actual = [100, 150, 200, 250, 300]
# Predicted values from a model
predicted = [110, 140, 210, 240, 310]

# Calculate MAE
mae = mean_absolute_error(actual, predicted)
# Calculate MSE
mse = mean_squared_error(actual, predicted)
# Calculate RMSE
rmse = np.sqrt(mse)

print(f"MAE: {mae:.2f}")
print(f"MSE: {mse:.2f}")
print(f"RMSE: {rmse:.2f}")

Important Notes

Lower values of MAE, MSE, and RMSE mean better predictions.

RMSE is more sensitive to large errors than MAE.

Always compare metrics on the same dataset to judge model improvements.

Summary

Time series evaluation metrics help measure prediction errors.

MAE, MSE, and RMSE are common and easy to use.

Use these metrics to improve and trust your forecasting models.