Time series evaluation metrics in ML Python - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to calculate the Mean Absolute Error (MAE) between true and predicted values.

ML Python

from sklearn.metrics import [1]
true = [3, -0.5, 2, 7]
pred = [2.5, 0.0, 2, 8]
mae = [1](true, pred)
print(mae)

Drag options to blanks, or click blank then click option'

Ar2_score

Bmean_absolute_error

Caccuracy_score

Dmean_squared_error

Attempts:

3 left

💡 Hint

Common Mistakes

Using mean_squared_error instead of mean_absolute_error

Using accuracy_score which is for classification

The Mean Absolute Error (MAE) measures the average absolute difference between true and predicted values. The function mean_absolute_error from sklearn.metrics computes this metric.

2fill in blank

medium

Complete the code to calculate the Root Mean Squared Error (RMSE) from the Mean Squared Error (MSE).

ML Python

from sklearn.metrics import mean_squared_error
import numpy as np
true = [3, -0.5, 2, 7]
pred = [2.5, 0.0, 2, 8]
mse = mean_squared_error(true, pred)
rmse = np.[1](mse)
print(rmse)

Drag options to blanks, or click blank then click option'

Asquare

Bexp

Clog

Dsqrt

Attempts:

3 left

💡 Hint

Common Mistakes

Using np.square instead of np.sqrt

Using np.log or np.exp which are unrelated

The Root Mean Squared Error (RMSE) is the square root of the Mean Squared Error (MSE). The np.sqrt function calculates the square root.

3fill in blank

hard

Fix the error in the code to calculate Mean Absolute Percentage Error (MAPE) manually.

ML Python

true = [100, 200, 300, 400]
pred = [110, 190, 310, 420]
mape = 100 * sum(abs((true[i] - pred[i]) / [1]) for i in range(len(true))) / len(true)
print(mape)

Drag options to blanks, or click blank then click option'

Asum(true)

Bpred[i]

Ctrue[i]

Dlen(true)

Attempts:

3 left

💡 Hint

Common Mistakes

Dividing by predicted value instead of true value

Dividing by sum or length instead of element-wise true value

Mean Absolute Percentage Error (MAPE) divides the absolute error by the true value at each point. So the denominator should be true[i].

4fill in blank

hard

Fill both blanks to create a dictionary of errors where keys are error names and values are their computed scores.

ML Python

from sklearn.metrics import mean_absolute_error, mean_squared_error
true = [1, 2, 3, 4]
pred = [1.1, 1.9, 3.2, 3.8]
errors = {
    'MAE': [1](true, pred),
    'MSE': [2](true, pred)
}
print(errors)

Drag options to blanks, or click blank then click option'

Amean_absolute_error

Bmean_squared_error

Cr2_score

Daccuracy_score

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up MAE and MSE function names

Using classification metrics like accuracy_score

The dictionary stores the Mean Absolute Error (MAE) and Mean Squared Error (MSE) calculated using their respective functions.

5fill in blank

hard

Fill all three blanks to create a dictionary comprehension that maps each error name to its score, filtering only errors with score less than 0.5.

ML Python

from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
true = [2, 4, 6, 8]
pred = [2.1, 3.9, 6.2, 7.8]
errors = {
    [1]: [2](true, pred)
    for [3] in ['mean_absolute_error', 'mean_squared_error', 'r2_score']
    if [2](true, pred) < 0.5
}
print(errors)

Drag options to blanks, or click blank then click option'

Aerror_name

Bglobals()[error_name]

Derror

Attempts:

3 left

💡 Hint

Common Mistakes

Using different variable names inconsistently

Trying to call functions without using globals() or similar

The dictionary comprehension uses error_name as key, calls the function by name using globals()[error_name], and iterates over error_name in the list of metric names. It filters errors with score less than 0.5.

Practice

1. Which metric measures the average absolute difference between predicted and actual values in time series forecasting?

easy

A. Mean Squared Error (MSE)

B. Mean Absolute Error (MAE)

C. Root Mean Squared Error (RMSE)

D. R-squared (Coefficient of Determination)

Solution

1. Step 1: Understand the definition of MAE

   MAE calculates the average of the absolute differences between predicted and actual values, showing average error size.

2. Step 2: Compare with other metrics

   MSE and RMSE square errors, while R-squared measures variance explained, not average error.

3. Final Answer:

   Mean Absolute Error (MAE) -> Option B

4. Quick Check:

   Average absolute difference = MAE [OK]

Hint: MAE uses absolute differences, no squaring involved [OK]

Common Mistakes:

• Confusing MAE with MSE or RMSE
• Thinking R-squared measures error size
• Assuming RMSE is the same as MAE

2. Which of the following is the correct formula for Root Mean Squared Error (RMSE) given errors \(e_i = y_i - \hat{y}_i\) for \(n\) points?

easy

A. RMSE = \(\sum_{i=1}^n e_i^2\)

B. RMSE = \(\frac{1}{n} \sum_{i=1}^n |e_i|\)

C. RMSE = \(\frac{1}{n} \sum_{i=1}^n e_i\)

D. RMSE = \(\sqrt{\frac{1}{n} \sum_{i=1}^n e_i^2}\)

Solution

1. Step 1: Recall RMSE formula

   RMSE is the square root of the average of squared errors, so it must include squaring, averaging, then square root.

2. Step 2: Check each option

   RMSE = \(\sqrt{\frac{1}{n} \sum_{i=1}^n e_i^2}\): \(\sqrt{\frac{1}{n} \sum_{i=1}^n e_i^2}\) matches the formula exactly. RMSE = \(\sum_{i=1}^n e_i^2\) misses averaging and root. RMSE = \(\frac{1}{n} \sum_{i=1}^n |e_i|\) is MAE. RMSE = \(\frac{1}{n} \sum_{i=1}^n e_i\) is mean error (not squared).

3. Final Answer:

   RMSE = \(\sqrt{\frac{1}{n} \sum_{i=1}^n e_i^2}\) -> Option D

4. Quick Check:

   RMSE = sqrt(mean squared errors) [OK]

Hint: RMSE = square root of average squared errors [OK]

Common Mistakes:

• Forgetting to take square root
• Using absolute errors instead of squared
• Not dividing by number of points

3. Given actual values \([3, 5, 2, 7]\) and predicted values \([2, 5, 4, 8]\), what is the Mean Squared Error (MSE)?

medium

A. 1.5

B. 1.25

C. 2.0

D. 0.75

Solution

1. Step 1: Calculate errors and square them

   Errors: 3-2=1, 5-5=0, 2-4=-2, 7-8=-1. Squared errors: 1, 0, 4, 1.

2. Step 2: Compute average of squared errors

   Sum = 1+0+4+1=6. Average = 6/4 = 1.5.

3. Final Answer:

   1.5 -> Option A

4. Quick Check:

   Sum squared errors / count = 1.5 [OK]

Hint: Square errors, sum, then divide by count [OK]

Common Mistakes:

• Using absolute errors instead of squared
• Forgetting to average over all points
• Mixing predicted and actual values

4. Identify the error in this Python code calculating MAE for time series predictions:

def mae(actual, predicted):
    errors = [a - p for a, p in zip(actual, predicted)]
    return sum(errors) / len(errors)

medium

A. Use multiplication instead of subtraction in errors

B. Divide by sum of errors instead of length

C. Errors should be absolute values before summing

D. No error, code is correct

Solution

1. Step 1: Analyze error calculation

   The code calculates errors as differences but does not take absolute values, which MAE requires.

2. Step 2: Understand MAE definition

   MAE is mean of absolute errors, so errors must be wrapped with abs() before summing.

3. Final Answer:

   Errors should be absolute values before summing -> Option C

4. Quick Check:

   MAE needs absolute errors [OK]

Hint: MAE sums absolute errors, not raw differences [OK]

Common Mistakes:

• Skipping absolute value in error calculation
• Dividing by wrong denominator
• Confusing MAE with MSE

5. You have two forecasting models evaluated on the same dataset. Model A has MAE=2.5 and RMSE=3.0, Model B has MAE=2.0 and RMSE=3.5. Which model is generally better and why?

hard

A. Model A, because lower RMSE means fewer large errors

B. Model B, because higher RMSE indicates better fit

C. Model B, because lower MAE means better average error

D. Model A, because MAE and RMSE must be equal for best model

Solution

1. Step 1: Interpret MAE and RMSE values

   Model B has lower MAE but higher RMSE, meaning it has better average error but more large errors. Model A has lower RMSE, indicating fewer large errors.

2. Step 2: Decide which metric matters more

   RMSE penalizes large errors more, so lower RMSE often means more reliable predictions without big mistakes.

3. Final Answer:

   Model A, because lower RMSE means fewer large errors -> Option A

4. Quick Check:

   Lower RMSE means fewer big errors [OK]

Hint: Lower RMSE means fewer big errors; prefer it if large errors matter [OK]

Common Mistakes:

• Choosing model with lower MAE ignoring RMSE
• Thinking higher RMSE is better
• Expecting MAE and RMSE to be equal