Evaluation metrics (RMSE, precision@k) in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does RMSE stand for and what does it measure?

RMSE stands for Root Mean Squared Error. It measures the average size of the errors between predicted values and actual values, giving more weight to larger errors.

beginner

Explain precision@k in simple terms.

Precision@k measures how many of the top k items recommended by a model are actually relevant or correct. It helps check the quality of recommendations.

intermediate

Why is RMSE sensitive to large errors?

Because RMSE squares the errors before averaging, bigger errors become much larger when squared, making RMSE more sensitive to those large mistakes.

beginner

How would you interpret a precision@5 score of 0.8?

It means that out of the top 5 items recommended, 4 are relevant or correct (since 0.8 × 5 = 4). So, 80% of the top 5 recommendations are good.

beginner

What kind of problems is RMSE commonly used for?

RMSE is commonly used for regression problems where the goal is to predict continuous numbers, like house prices or temperatures.

What does RMSE measure in a model's predictions?

A Number of true positives

B Percentage of correct classifications

C Average size of prediction errors

D Speed of the model

RMSE measures the average size of the errors between predicted and actual values.

Precision@k is useful for evaluating which type of model?

A Recommendation or ranking models

B Regression models

C Classification models

D Clustering models

Precision@k evaluates how many of the top k recommended items are relevant, so it is used for recommendation or ranking models.

If RMSE is very high, what does it mean about the model's predictions?

A Predictions have large errors

B Predictions are very close to actual values

C Model is very fast

D Model has perfect accuracy

A high RMSE means the model's predictions have large errors compared to actual values.

Which of these is true about precision@k?

A It measures error size in regression

B It counts how many relevant items are in the top k results

C It measures the speed of training

D It is used only for binary classification

Precision@k counts how many relevant items appear in the top k results of a recommendation or ranking.

Why do we square errors when calculating RMSE?

A To count errors as zero

B To ignore small errors

C To make errors negative

D To give more weight to larger errors

Squaring errors makes larger errors count more, so RMSE is sensitive to big mistakes.

Describe what RMSE is and how it helps evaluate a model.

Think about how RMSE shows the average size of prediction mistakes.

Explain precision@k and why it is useful for recommendation systems.

Consider how we check if the top suggestions are actually good.