Time series evaluation metrics in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Mean Absolute Error (MAE) in Time Series

Which statement best describes the Mean Absolute Error (MAE) when used to evaluate time series predictions?

A MAE calculates the average of the absolute differences between predicted and actual values, treating all errors equally regardless of direction.

B MAE measures the average of the squared differences between predicted and actual values, penalizing larger errors more heavily.

C MAE measures the correlation between predicted and actual values to assess prediction accuracy.

D MAE is the percentage of predictions that exactly match the actual values in the time series.

💡 Hint

Think about how MAE treats errors without squaring them.

MAE calculates the average absolute difference between predicted and actual values, giving equal weight to all errors regardless of their sign.

❓ Predict Output

intermediate

Calculate RMSE for a Time Series Prediction

What is the output of the following Python code that calculates the Root Mean Squared Error (RMSE) for given true and predicted values?

ML Python

import numpy as np
true = np.array([3, -0.5, 2, 7])
pred = np.array([2.5, 0.0, 2, 8])
rmse = np.sqrt(np.mean((true - pred) ** 2))
print(round(rmse, 3))

A 0.750

B 1.118

C 0.612

D 0.957

💡 Hint

Calculate squared errors, average them, then take the square root.

Squared errors: (0.5², (-0.5)², 0², (-1)²) = (0.25, 0.25, 0, 1). Mean = 1.5/4 = 0.375. RMSE = √0.375 ≈ 0.612 (option C).

❓ Model Choice

advanced

Choosing the Best Metric for Skewed Time Series Data

You have a time series dataset with many small values and a few very large spikes. Which evaluation metric is best to use to avoid the large spikes dominating the error measurement?

A R-squared (Coefficient of Determination)

B Root Mean Squared Error (RMSE)

C Mean Squared Error (MSE)

D Mean Absolute Error (MAE)

💡 Hint

Consider which metric is less sensitive to large errors.

MAE treats all errors equally by using absolute values, so large spikes do not disproportionately affect the metric, unlike MSE or RMSE which square errors.

❓ Metrics

advanced

Interpreting Mean Absolute Percentage Error (MAPE)

Given the true values [100, 200, 300] and predicted values [110, 190, 310], what is the Mean Absolute Percentage Error (MAPE) expressed as a percentage?

ML Python

true = [100, 200, 300]
pred = [110, 190, 310]
errors = [abs(t - p) / t for t, p in zip(true, pred)]
mape = sum(errors) / len(errors) * 100
print(round(mape, 2))

A 8.33

B 6.11

C 7.14

D 5.56

💡 Hint

Calculate absolute percentage errors for each point, then average.

Errors: |100-110|/100=0.1 (10%), |200-190|/200=0.05 (5%), |300-310|/300≈0.0333 (3.33%). Average ≈0.0611 or 6.11% (option B).

🔧 Debug

expert

Identify the Error in Time Series Metric Calculation Code

What error does the following Python code raise when calculating Mean Absolute Error (MAE) for time series predictions?

ML Python

true = [1, 2, 3, 4]
pred = [1, 2, 3]
mae = sum(abs(t - p) for t, p in zip(true, pred)) / len(true)
print(mae)

A No error; output is 0.0

B No error; output is 0.25

C IndexError because lists have different lengths

D ZeroDivisionError because denominator is zero

💡 Hint

Check how zip works with lists of different lengths.

zip stops at the shortest list (pred), so only pairs (1,1), (2,2), (3,3) with errors 0,0,0. Sum=0 / len(true)=4 = 0.0. No error.

Practice

1. Which metric measures the average absolute difference between predicted and actual values in time series forecasting?

easy

A. Mean Squared Error (MSE)

B. Mean Absolute Error (MAE)

C. Root Mean Squared Error (RMSE)

D. R-squared (Coefficient of Determination)

Solution

1. Step 1: Understand the definition of MAE

   MAE calculates the average of the absolute differences between predicted and actual values, showing average error size.

2. Step 2: Compare with other metrics

   MSE and RMSE square errors, while R-squared measures variance explained, not average error.

3. Final Answer:

   Mean Absolute Error (MAE) -> Option B

4. Quick Check:

   Average absolute difference = MAE [OK]

Hint: MAE uses absolute differences, no squaring involved [OK]

Common Mistakes:

• Confusing MAE with MSE or RMSE
• Thinking R-squared measures error size
• Assuming RMSE is the same as MAE

2. Which of the following is the correct formula for Root Mean Squared Error (RMSE) given errors \(e_i = y_i - \hat{y}_i\) for \(n\) points?

easy

A. RMSE = \(\sum_{i=1}^n e_i^2\)

B. RMSE = \(\frac{1}{n} \sum_{i=1}^n |e_i|\)

C. RMSE = \(\frac{1}{n} \sum_{i=1}^n e_i\)

D. RMSE = \(\sqrt{\frac{1}{n} \sum_{i=1}^n e_i^2}\)

Solution

1. Step 1: Recall RMSE formula

   RMSE is the square root of the average of squared errors, so it must include squaring, averaging, then square root.

2. Step 2: Check each option

   RMSE = \(\sqrt{\frac{1}{n} \sum_{i=1}^n e_i^2}\): \(\sqrt{\frac{1}{n} \sum_{i=1}^n e_i^2}\) matches the formula exactly. RMSE = \(\sum_{i=1}^n e_i^2\) misses averaging and root. RMSE = \(\frac{1}{n} \sum_{i=1}^n |e_i|\) is MAE. RMSE = \(\frac{1}{n} \sum_{i=1}^n e_i\) is mean error (not squared).

3. Final Answer:

   RMSE = \(\sqrt{\frac{1}{n} \sum_{i=1}^n e_i^2}\) -> Option D

4. Quick Check:

   RMSE = sqrt(mean squared errors) [OK]

Hint: RMSE = square root of average squared errors [OK]

Common Mistakes:

• Forgetting to take square root
• Using absolute errors instead of squared
• Not dividing by number of points

3. Given actual values \([3, 5, 2, 7]\) and predicted values \([2, 5, 4, 8]\), what is the Mean Squared Error (MSE)?

medium

A. 1.5

B. 1.25

C. 2.0

D. 0.75

Solution

1. Step 1: Calculate errors and square them

   Errors: 3-2=1, 5-5=0, 2-4=-2, 7-8=-1. Squared errors: 1, 0, 4, 1.

2. Step 2: Compute average of squared errors

   Sum = 1+0+4+1=6. Average = 6/4 = 1.5.

3. Final Answer:

   1.5 -> Option A

4. Quick Check:

   Sum squared errors / count = 1.5 [OK]

Hint: Square errors, sum, then divide by count [OK]

Common Mistakes:

• Using absolute errors instead of squared
• Forgetting to average over all points
• Mixing predicted and actual values

4. Identify the error in this Python code calculating MAE for time series predictions:

def mae(actual, predicted):
    errors = [a - p for a, p in zip(actual, predicted)]
    return sum(errors) / len(errors)

medium

A. Use multiplication instead of subtraction in errors

B. Divide by sum of errors instead of length

C. Errors should be absolute values before summing

D. No error, code is correct

Solution

1. Step 1: Analyze error calculation

   The code calculates errors as differences but does not take absolute values, which MAE requires.

2. Step 2: Understand MAE definition

   MAE is mean of absolute errors, so errors must be wrapped with abs() before summing.

3. Final Answer:

   Errors should be absolute values before summing -> Option C

4. Quick Check:

   MAE needs absolute errors [OK]

Hint: MAE sums absolute errors, not raw differences [OK]

Common Mistakes:

• Skipping absolute value in error calculation
• Dividing by wrong denominator
• Confusing MAE with MSE

5. You have two forecasting models evaluated on the same dataset. Model A has MAE=2.5 and RMSE=3.0, Model B has MAE=2.0 and RMSE=3.5. Which model is generally better and why?

hard

A. Model A, because lower RMSE means fewer large errors

B. Model B, because higher RMSE indicates better fit

C. Model B, because lower MAE means better average error

D. Model A, because MAE and RMSE must be equal for best model

Solution

1. Step 1: Interpret MAE and RMSE values

   Model B has lower MAE but higher RMSE, meaning it has better average error but more large errors. Model A has lower RMSE, indicating fewer large errors.

2. Step 2: Decide which metric matters more

   RMSE penalizes large errors more, so lower RMSE often means more reliable predictions without big mistakes.

3. Final Answer:

   Model A, because lower RMSE means fewer large errors -> Option A

4. Quick Check:

   Lower RMSE means fewer big errors [OK]

Hint: Lower RMSE means fewer big errors; prefer it if large errors matter [OK]

Common Mistakes:

• Choosing model with lower MAE ignoring RMSE
• Thinking higher RMSE is better
• Expecting MAE and RMSE to be equal