Evaluation metrics (RMSE, precision@k) in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Calculate RMSE for given predictions and true values

Given the true values and predicted values below, what is the RMSE (Root Mean Squared Error)?

ML Python

import numpy as np
true_values = np.array([3, -0.5, 2, 7])
predictions = np.array([2.5, 0.0, 2, 8])
rmse = np.sqrt(np.mean((true_values - predictions) ** 2))
print(round(rmse, 3))

A 0.500

B 0.750

C 1.000

D 0.612

💡 Hint

RMSE is the square root of the average of squared differences between true and predicted values.

RMSE measures how far predictions are from true values on average. Here, differences squared are [0.25, 0.25, 0, 1], average is 0.375, square root is about 0.612.

🧠 Conceptual

intermediate

Understanding Precision@k in recommendation systems

In a recommendation system, what does precision@3 measure?

A The fraction of the top 3 recommended items that are relevant to the user

B The fraction of all relevant items recommended to the user

C The fraction of users who received at least 3 relevant recommendations

D The average rating of the top 3 recommended items

💡 Hint

Precision@k focuses on the top k items recommended and how many are relevant.

Precision@k counts how many of the top k recommended items are actually relevant, divided by k. It shows accuracy of top recommendations.

❓ Metrics

advanced

Choosing the right metric for regression error

Which metric is more sensitive to large errors in predictions?

A Mean Absolute Percentage Error (MAPE)

B Mean Absolute Error (MAE)

C Root Mean Squared Error (RMSE)

D R-squared (Coefficient of Determination)

💡 Hint

Think about how squaring errors affects large mistakes.

RMSE squares the errors before averaging, so large errors have more impact than in MAE which uses absolute values.

🔧 Debug

advanced

Identify the error in precision@k calculation code

What error does the following code produce when calculating precision@k? ```python relevant_items = {1, 3, 5, 7} recommended_items = [2, 3, 4, 5] k = 3 precision_at_k = len(set(recommended_items[:k]) & relevant_items) / len(recommended_items) print(round(precision_at_k, 2)) ```

A Incorrect precision value because denominator should be k, not total recommended items

B ZeroDivisionError because len(recommended_items) is zero

C TypeError because set intersection is invalid between list and set

D SyntaxError due to missing colon

💡 Hint

Precision@k divides by k, not total recommended items length.

The denominator should be k (number of top items considered), but code divides by total recommended items length, giving wrong precision.

❓ Model Choice

expert

Selecting evaluation metric for imbalanced classification with ranking

You have a highly imbalanced dataset and want to evaluate a model that ranks positive samples higher than negatives. Which metric is best suited?

A Accuracy

B Precision@k

C Root Mean Squared Error (RMSE)

D Mean Squared Error (MSE)

💡 Hint

Think about metrics that focus on top-ranked relevant items in imbalanced data.

Precision@k measures how many of the top k ranked items are relevant, which is useful in imbalanced data where overall accuracy can be misleading.