User-based vs item-based in ML Python - Metrics Comparison

Metrics & Evaluation - User-based vs item-based

Which metric matters for User-based vs item-based and WHY

In recommendation systems, the key metrics are Precision, Recall, and F1-score. These tell us how well the system suggests items users actually like.

Precision shows how many recommended items are truly relevant. Recall shows how many relevant items were found out of all possible relevant items. F1-score balances both.

We use these because recommendations must be both accurate (precision) and complete (recall) to keep users happy.

Confusion matrix for recommendation

      |---------------------------|
      |           | Relevant      | Not Relevant |
      |-----------|--------------|-------------|
      | Recommended | True Positive (TP) | False Positive (FP) |
      | Not Recommended | False Negative (FN) | True Negative (TN) |
      |---------------------------|
    

For example, if a system recommends 10 items, 7 are liked (TP), 3 are not (FP). If there are 5 liked items not recommended (FN), and many irrelevant items not recommended (TN).

Precision vs Recall tradeoff with examples

User-based filtering often has higher recall because it finds items liked by similar users, but may recommend some irrelevant items, lowering precision.

Item-based filtering tends to have higher precision because it recommends items similar to what the user liked, but may miss some relevant items, lowering recall.

Example: For a movie app, user-based may suggest more diverse movies (higher recall), but some may not fit user taste (lower precision). Item-based suggests movies very similar to watched ones (higher precision), but fewer new discoveries (lower recall).

What good vs bad metric values look like

Good metrics: Precision and recall both above 0.7 means recommendations are mostly relevant and cover many liked items.

Bad metrics: Precision below 0.4 means many irrelevant items are recommended. Recall below 0.3 means many liked items are missed.

For example, a user-based system with precision 0.75 and recall 0.8 is good. An item-based system with precision 0.85 but recall 0.25 may miss many relevant items.

Common pitfalls in metrics

• Accuracy paradox: High accuracy can be misleading if most items are irrelevant and not recommended.
• Data leakage: Using future user data in training inflates metrics falsely.
• Overfitting: Model fits training users/items too closely but fails on new users/items, causing poor real-world metrics.
• Ignoring diversity: High precision but recommending very similar items can bore users.

Self-check question

Your recommendation model has 98% accuracy but only 12% recall on relevant items. Is it good for production?

Answer: No. The high accuracy is misleading because most items are irrelevant and not recommended. The very low recall means the model misses most items users would like, so it fails to provide useful recommendations.

Key Result

Precision and recall are key to evaluate user-based vs item-based recommendation quality, balancing relevance and coverage.

Practice

1. Which of the following best describes user-based recommendation?

easy

A. Suggest items liked by users similar to you

B. Suggest items similar to what you liked

C. Suggest random popular items

D. Suggest items based on item price

Solution

1. Step 1: Understand user-based recommendation

   User-based recommendation finds users similar to you and suggests what they liked.

2. Step 2: Compare options

   Suggest items liked by users similar to you matches this definition exactly, while others describe different or unrelated methods.

3. Final Answer:

   Suggest items liked by users similar to you -> Option A

4. Quick Check:

   User-based = Suggest items liked by similar users [OK]

Hint: User-based = find similar users, recommend their items [OK]

Common Mistakes:

• Confusing user-based with item-based
• Thinking user-based recommends similar items
• Choosing popular or random items as user-based

2. Which code snippet correctly represents item-based recommendation logic?

easy

A. recommend_items = find_similar_users(user).liked_items()

B. recommend_items = find_similar_items(user_liked_items)

C. recommend_items = random_popular_items()

D. recommend_items = items_sorted_by_price()

Solution

1. Step 1: Identify item-based recommendation code

   Item-based recommends items similar to those the user liked, so code should find similar items.

2. Step 2: Match code to logic

   recommend_items = find_similar_items(user_liked_items) calls find_similar_items(user_liked_items), which fits item-based logic. Others do not.

3. Final Answer:

   <code>recommend_items = find_similar_items(user_liked_items)</code> -> Option B

4. Quick Check:

   Item-based = find similar items [OK]

Hint: Item-based code finds similar items to user's liked items [OK]

Common Mistakes:

• Choosing user-based code for item-based question
• Confusing random or price sorting with item similarity
• Ignoring the input to similarity function

3. Given this code snippet for user-based recommendation:

user_similarities = {'User1': 0.9, 'User2': 0.8}
user_likes = {'User1': ['ItemA', 'ItemB'], 'User2': ['ItemB', 'ItemC']}
recommend = set()
for user, sim in user_similarities.items():
    if sim > 0.85:
        recommend.update(user_likes[user])
print(sorted(recommend))

What is the output?

medium

A. ['ItemA', 'ItemB', 'ItemC']

B. ['ItemB', 'ItemC']

C. ['ItemA', 'ItemB']

D. []

Solution

1. Step 1: Identify users with similarity > 0.85

   Only 'User1' has similarity 0.9 > 0.85, so only their liked items are added.

2. Step 2: Collect items liked by 'User1'

   'User1' likes ['ItemA', 'ItemB'], so these are added to the recommendation set.

3. Final Answer:

   ['ItemA', 'ItemB'] -> Option C

4. Quick Check:

   Similarity filter picks User1's items [OK]

Hint: Only users with similarity > 0.85 contribute items [OK]

Common Mistakes:

• Including items from User2 who has similarity 0.8
• Not sorting output before printing
• Confusing set update with append

4. Find the error in this item-based recommendation code:

user_liked = ['ItemA', 'ItemB']
def find_similar_items(items):
    return ['ItemC', 'ItemD']
recommend = find_similar_items(user_liked)
print(recommend)

What is the main problem?

medium

A. Syntax error in function definition

B. Print statement missing parentheses

C. Variable user_liked is not defined

D. Function returns unrelated items, ignoring input

Solution

1. Step 1: Check function behavior

   The function find_similar_items ignores its input and returns fixed items.

2. Step 2: Identify why this is a problem

   Item-based recommendation should return items similar to input, but this function does not use input at all.

3. Final Answer:

   Function returns unrelated items, ignoring input -> Option D

4. Quick Check:

   Function must use input to find similar items [OK]

Hint: Check if function uses input parameter correctly [OK]

Common Mistakes:

• Thinking code has syntax errors when it doesn't
• Ignoring that function ignores input
• Confusing variable names or print syntax

5. You want to build a hybrid recommendation system combining user-based and item-based methods. Which approach best fits this goal?

hard

A. Combine scores from similar users and similar items for each candidate

B. Use only user-based recommendations to simplify model

C. Use only item-based recommendations to reduce computation

D. Randomly pick recommendations from either user or item-based

Solution

1. Step 1: Understand hybrid recommendation

   Hybrid systems combine strengths of user-based and item-based methods to improve accuracy.

2. Step 2: Identify best combination method

   Combining scores from both similar users and similar items for each candidate item leverages both perspectives effectively.

3. Final Answer:

   Combine scores from similar users and similar items for each candidate -> Option A

4. Quick Check:

   Hybrid = combine user and item scores [OK]

Hint: Hybrid = combine user and item recommendation scores [OK]

Common Mistakes:

• Choosing only one method loses hybrid benefits
• Random picking reduces recommendation quality
• Ignoring score combination logic