Content-based filtering in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - Content-based filtering

Which metric matters for Content-based filtering and WHY

Content-based filtering recommends items similar to what a user liked before. The key metric is Precision. This tells us how many recommended items are actually relevant to the user. High precision means the system suggests mostly things the user will like, avoiding annoying wrong suggestions.

Recall is also important. It measures how many of the relevant items the system finds. High recall means the system finds many good matches, not missing useful recommendations.

Balancing precision and recall is important. The F1 score combines both into one number to check overall quality.

Confusion matrix for Content-based filtering

      |-----------------------------|
      |           | Predicted       |
      | Actual    | Relevant | Not  |
      |           |         | Relevant |
      |-----------------------------|
      | Relevant  |   TP    |   FN   |
      | Not       |   FP    |   TN   |
      | Relevant  |         |        |
      |-----------------------------|

      TP = Recommended and relevant (good)
      FP = Recommended but not relevant (bad)
      FN = Not recommended but relevant (missed)
      TN = Not recommended and not relevant (neutral)
    

Precision vs Recall tradeoff with examples

If the system recommends fewer items, it may have high precision (mostly good suggestions) but low recall (missing many relevant items).

If it recommends many items, it may have high recall (finding many relevant items) but low precision (more wrong suggestions).

Example: A movie recommender that suggests only 2 movies might have high precision if both are liked, but low recall if the user likes 10 movies total. Suggesting 20 movies might catch all 10 liked ones (high recall) but also include many disliked ones (low precision).

What good vs bad metric values look like

Good: Precision and recall both above 0.7 means the system suggests mostly relevant items and finds many of them.

Bad: Precision below 0.3 means many wrong suggestions, annoying users. Recall below 0.3 means missing most relevant items, making recommendations useless.

F1 score below 0.4 usually means the system needs improvement.

Common pitfalls in metrics for Content-based filtering

• Accuracy paradox: Accuracy can be misleading if most items are irrelevant. A system that never recommends can have high accuracy but no value.
• Data leakage: Using future user preferences in training can inflate metrics unrealistically.
• Overfitting: The system may recommend only items very similar to past likes, missing diversity and hurting recall.

Self-check question

Your content-based filtering model has 98% accuracy but only 12% recall on relevant items. Is it good for production? Why or why not?

Answer: No, it is not good. The high accuracy is misleading because most items are irrelevant, so the model is mostly not recommending anything. The very low recall means it misses almost all relevant items, so users get very few useful recommendations.

Key Result

Precision and recall are key metrics; high precision ensures relevant recommendations, high recall ensures many relevant items are found.

Practice

1. What is the main idea behind content-based filtering in recommendation systems?

easy

A. It recommends items based on what other users liked.

B. It recommends items similar to what the user liked before based on item features.

C. It randomly suggests items without any user data.

D. It recommends items based on the most popular items overall.

Solution

1. Step 1: Understand content-based filtering concept

   Content-based filtering uses the features of items a user liked to find similar items.

2. Step 2: Compare with other recommendation methods

   Unlike collaborative filtering, it does not rely on other users' preferences but on item features.

3. Final Answer:

   It recommends items similar to what the user liked before based on item features. -> Option B

4. Quick Check:

   Content-based filtering = Recommend similar items [OK]

Hint: Focus on item features and user history for content-based filtering [OK]

Common Mistakes:

• Confusing content-based with collaborative filtering
• Thinking recommendations are random
• Assuming popularity drives recommendations

2. Which of the following is the correct way to represent item features for content-based filtering in Python?

easy

A. item_features = [1, 2, 3, 4]

B. item_features = ['item1', 'item2', 'item3']

C. item_features = {'item1': 'popular', 'item2': 'new'}

D. item_features = {'item1': [0.2, 0.8], 'item2': [0.5, 0.5]}

Solution

1. Step 1: Identify correct data structure for features

   Features should be numeric vectors representing item properties, stored in a dictionary keyed by item IDs.

2. Step 2: Check each option

   item_features = {'item1': [0.2, 0.8], 'item2': [0.5, 0.5]} uses a dictionary with lists of numbers, suitable for feature vectors. Others are lists or strings, not suitable for numeric features.

3. Final Answer:

   item_features = {'item1': [0.2, 0.8], 'item2': [0.5, 0.5]} -> Option D

4. Quick Check:

   Features as numeric vectors = item_features = {'item1': [0.2, 0.8], 'item2': [0.5, 0.5]} [OK]

Hint: Use dictionaries with numeric lists for item features [OK]

Common Mistakes:

• Using strings instead of numeric vectors
• Using simple lists without keys
• Confusing item labels with features

3. Given the following Python code for content-based filtering, what will be the output?

import numpy as np

item_features = {'A': np.array([1, 0]), 'B': np.array([0, 1]), 'C': np.array([1, 1])}
user_likes = np.array([1, 0])
similarities = {item: np.dot(user_likes, features) for item, features in item_features.items()}
recommended = max(similarities, key=similarities.get)
print(recommended)

medium

A. B

B. C

C. A

D. Error

Solution

1. Step 1: Calculate dot products for similarity

   Dot product with user_likes [1,0]: - A: [1,0]·[1,0] = 1 - B: [1,0]·[0,1] = 0 - C: [1,0]·[1,1] = 1

2. Step 2: Find item with max similarity

   Items A and C both have similarity 1, but max() returns the first max key, which is 'A'.

3. Final Answer:

   A -> Option C

4. Quick Check:

   Max dot product = 'A' [OK]

Hint: Dot product similarity picks highest matching feature vector [OK]

Common Mistakes:

• Assuming 'C' is chosen because it has more features
• Confusing dot product with sum
• Expecting an error due to dictionary comprehension

4. Identify the error in this content-based filtering code snippet and select the fix:

item_features = {'X': [1, 0], 'Y': [0, 1]}
user_profile = [1, 0]
similarities = {}
for item in item_features:
    similarities[item] = user_profile * item_features[item]
print(similarities)

medium

A. Replace '*' with a dot product function like numpy.dot

B. Change user_profile and item_features values to strings

C. Use '+' instead of '*' to combine lists

D. No error, code runs fine

Solution

1. Step 1: Understand the operation on lists

   Multiplying two Python lists with '*' is invalid and causes a TypeError.

2. Step 2: Correct the operation to compute similarity

   Use numpy.dot or equivalent to compute dot product between vectors for similarity.

3. Final Answer:

   Replace '*' with a dot product function like numpy.dot -> Option A

4. Quick Check:

   List multiplication invalid, use dot product [OK]

Hint: Use dot product for vector similarity, not '*' on lists [OK]

Common Mistakes:

• Trying to multiply lists directly
• Using '+' which concatenates lists, not similarity
• Ignoring error messages

5. You want to recommend movies to a user using content-based filtering. The user liked movies with genres ['Action', 'Adventure']. You have a movie database with genre tags. Which approach best personalizes recommendations?

hard

A. Calculate similarity by counting overlapping genres between user liked movies and candidate movies.

B. Recommend the most popular movies regardless of genre.

C. Recommend movies liked by other users who watched 'Action' movies.

D. Randomly select movies from the database.

Solution

1. Step 1: Understand content-based filtering with genre tags

   Content-based filtering uses item features (genres) to find similar items to what the user liked.

2. Step 2: Choose method that compares genre overlap

   Counting overlapping genres directly measures similarity between user preferences and candidate movies.

3. Final Answer:

   Calculate similarity by counting overlapping genres between user liked movies and candidate movies. -> Option A

4. Quick Check:

   Genre overlap similarity = Calculate similarity by counting overlapping genres between user liked movies and candidate movies. [OK]

Hint: Match user liked genres with movie genres for recommendations [OK]

Common Mistakes:

• Using popularity instead of user preferences
• Confusing collaborative filtering with content-based
• Random recommendations ignoring user data