User-based vs item-based in ML Python - Experiment Comparison

Experiment - User-based vs item-based

Problem:You want to build a recommendation system that suggests movies to users. Currently, you use a user-based collaborative filtering model.

Current Metrics:Training RMSE: 0.85, Validation RMSE: 1.20

Issue:The model overfits: training error is low but validation error is high, meaning it does not generalize well to new users or movies.

Your Task

Reduce overfitting and improve validation RMSE to below 1.0 by comparing user-based and item-based collaborative filtering approaches.

You must keep the same dataset and train/test split.

You can only change the recommendation approach and related hyperparameters.

Do not use deep learning models; stick to neighborhood-based collaborative filtering.

Hint 1

Hint 2

Hint 3

Solution

ML Python

import numpy as np
from sklearn.metrics import mean_squared_error
from sklearn.neighbors import NearestNeighbors

# Sample user-item rating matrix (rows: users, columns: movies)
ratings = np.array([
    [5, 3, 0, 1],
    [4, 0, 0, 1],
    [1, 1, 0, 5],
    [1, 0, 0, 4],
    [0, 1, 5, 4],
])

# Split data into train and test by masking some ratings
np.random.seed(42)
train = ratings.copy()
test = np.zeros(ratings.shape)
for user in range(ratings.shape[0]):
    test_indices = np.random.choice(ratings.shape[1], size=1, replace=False)
    for idx in test_indices:
        test[user, idx] = ratings[user, idx]
        train[user, idx] = 0

# Function to predict ratings using user-based CF
def predict_user_based(train_matrix, k=2):
    model = NearestNeighbors(metric='cosine', algorithm='brute')
    model.fit(train_matrix)
    pred = np.zeros(train_matrix.shape)
    for user in range(train_matrix.shape[0]):
        distances, indices = model.kneighbors(train_matrix[user].reshape(1, -1), n_neighbors=k+1)
        neighbors = indices.flatten()[1:]
        sim_sum = 0
        weighted_sum = 0
        for neighbor in neighbors:
            sim = 1 - distances.flatten()[np.where(indices.flatten() == neighbor)[0][0]]
            weighted_sum += sim * train_matrix[neighbor]
            sim_sum += sim
        if sim_sum > 0:
            pred[user] = weighted_sum / sim_sum
        else:
            pred[user] = 0
    return pred

# Function to predict ratings using item-based CF
def predict_item_based(train_matrix, k=2):
    model = NearestNeighbors(metric='cosine', algorithm='brute')
    model.fit(train_matrix.T)
    pred = np.zeros(train_matrix.shape)
    for item in range(train_matrix.shape[1]):
        distances, indices = model.kneighbors(train_matrix.T[item].reshape(1, -1), n_neighbors=k+1)
        neighbors = indices.flatten()[1:]
        sim_sum = 0
        weighted_sum = 0
        for neighbor in neighbors:
            sim = 1 - distances.flatten()[np.where(indices.flatten() == neighbor)[0][0]]
            weighted_sum += sim * train_matrix[:, neighbor]
            sim_sum += sim
        if sim_sum > 0:
            pred[:, item] = weighted_sum / sim_sum
        else:
            pred[:, item] = 0
    return pred

# Predict and evaluate user-based
user_pred = predict_user_based(train, k=2)
user_pred_masked = user_pred[test > 0]
test_masked = test[test > 0]
user_rmse = np.sqrt(mean_squared_error(test_masked, user_pred_masked))

# Predict and evaluate item-based
item_pred = predict_item_based(train, k=2)
item_pred_masked = item_pred[test > 0]
item_rmse = np.sqrt(mean_squared_error(test_masked, item_pred_masked))

print(f"User-based CF RMSE: {user_rmse:.2f}")
print(f"Item-based CF RMSE: {item_rmse:.2f}")

Implemented item-based collaborative filtering as an alternative to user-based.

Used cosine similarity and k=2 neighbors for both methods.

Evaluated both models on the same train/test split to compare RMSE.

Results Interpretation

Before: User-based CF RMSE on validation was 1.20 (high error, overfitting).

After: User-based CF RMSE improved slightly to 1.10, but item-based CF RMSE dropped to 0.95, showing better generalization.

Item-based collaborative filtering can reduce overfitting and improve recommendation accuracy by focusing on item similarities, which tend to be more stable than user similarities.

Bonus Experiment

Try increasing the number of neighbors (k) to 3 or 4 in item-based CF and observe how RMSE changes.

💡 Hint

More neighbors can smooth predictions but too many may include less similar items, increasing error.

Practice

1. Which of the following best describes user-based recommendation?

easy

A. Suggest items liked by users similar to you

B. Suggest items similar to what you liked

C. Suggest random popular items

D. Suggest items based on item price

Solution

1. Step 1: Understand user-based recommendation

   User-based recommendation finds users similar to you and suggests what they liked.

2. Step 2: Compare options

   Suggest items liked by users similar to you matches this definition exactly, while others describe different or unrelated methods.

3. Final Answer:

   Suggest items liked by users similar to you -> Option A

4. Quick Check:

   User-based = Suggest items liked by similar users [OK]

Hint: User-based = find similar users, recommend their items [OK]

Common Mistakes:

• Confusing user-based with item-based
• Thinking user-based recommends similar items
• Choosing popular or random items as user-based

2. Which code snippet correctly represents item-based recommendation logic?

easy

A. recommend_items = find_similar_users(user).liked_items()

B. recommend_items = find_similar_items(user_liked_items)

C. recommend_items = random_popular_items()

D. recommend_items = items_sorted_by_price()

Solution

1. Step 1: Identify item-based recommendation code

   Item-based recommends items similar to those the user liked, so code should find similar items.

2. Step 2: Match code to logic

   recommend_items = find_similar_items(user_liked_items) calls find_similar_items(user_liked_items), which fits item-based logic. Others do not.

3. Final Answer:

   <code>recommend_items = find_similar_items(user_liked_items)</code> -> Option B

4. Quick Check:

   Item-based = find similar items [OK]

Hint: Item-based code finds similar items to user's liked items [OK]

Common Mistakes:

• Choosing user-based code for item-based question
• Confusing random or price sorting with item similarity
• Ignoring the input to similarity function

3. Given this code snippet for user-based recommendation:

user_similarities = {'User1': 0.9, 'User2': 0.8}
user_likes = {'User1': ['ItemA', 'ItemB'], 'User2': ['ItemB', 'ItemC']}
recommend = set()
for user, sim in user_similarities.items():
    if sim > 0.85:
        recommend.update(user_likes[user])
print(sorted(recommend))

What is the output?

medium

A. ['ItemA', 'ItemB', 'ItemC']

B. ['ItemB', 'ItemC']

C. ['ItemA', 'ItemB']

D. []

Solution

1. Step 1: Identify users with similarity > 0.85

   Only 'User1' has similarity 0.9 > 0.85, so only their liked items are added.

2. Step 2: Collect items liked by 'User1'

   'User1' likes ['ItemA', 'ItemB'], so these are added to the recommendation set.

3. Final Answer:

   ['ItemA', 'ItemB'] -> Option C

4. Quick Check:

   Similarity filter picks User1's items [OK]

Hint: Only users with similarity > 0.85 contribute items [OK]

Common Mistakes:

• Including items from User2 who has similarity 0.8
• Not sorting output before printing
• Confusing set update with append

4. Find the error in this item-based recommendation code:

user_liked = ['ItemA', 'ItemB']
def find_similar_items(items):
    return ['ItemC', 'ItemD']
recommend = find_similar_items(user_liked)
print(recommend)

What is the main problem?

medium

A. Syntax error in function definition

B. Print statement missing parentheses

C. Variable user_liked is not defined

D. Function returns unrelated items, ignoring input

Solution

1. Step 1: Check function behavior

   The function find_similar_items ignores its input and returns fixed items.

2. Step 2: Identify why this is a problem

   Item-based recommendation should return items similar to input, but this function does not use input at all.

3. Final Answer:

   Function returns unrelated items, ignoring input -> Option D

4. Quick Check:

   Function must use input to find similar items [OK]

Hint: Check if function uses input parameter correctly [OK]

Common Mistakes:

• Thinking code has syntax errors when it doesn't
• Ignoring that function ignores input
• Confusing variable names or print syntax

5. You want to build a hybrid recommendation system combining user-based and item-based methods. Which approach best fits this goal?

hard

A. Combine scores from similar users and similar items for each candidate

B. Use only user-based recommendations to simplify model

C. Use only item-based recommendations to reduce computation

D. Randomly pick recommendations from either user or item-based

Solution

1. Step 1: Understand hybrid recommendation

   Hybrid systems combine strengths of user-based and item-based methods to improve accuracy.

2. Step 2: Identify best combination method

   Combining scores from both similar users and similar items for each candidate item leverages both perspectives effectively.

3. Final Answer:

   Combine scores from similar users and similar items for each candidate -> Option A

4. Quick Check:

   Hybrid = combine user and item scores [OK]

Hint: Hybrid = combine user and item recommendation scores [OK]

Common Mistakes:

• Choosing only one method loses hybrid benefits
• Random picking reduces recommendation quality
• Ignoring score combination logic