Why recommendations drive engagement in ML Python - Experiment to Prove It

Experiment - Why recommendations drive engagement

Problem:We want to understand how a recommendation system can increase user engagement on a website. Currently, the model predicts user clicks on recommended items with 90% accuracy on training data but only 70% on validation data.

Current Metrics:Training accuracy: 90%, Validation accuracy: 70%, Training loss: 0.25, Validation loss: 0.65

Issue:The model is overfitting. It performs well on training data but poorly on new, unseen data, meaning recommendations may not generalize well to real users.

Your Task

Reduce overfitting so that validation accuracy improves to at least 80% while keeping training accuracy below 88%. This will help the recommendation system better predict user engagement.

You can only change model architecture and training hyperparameters.

Do not change the dataset or add new data.

Keep the model training time reasonable (under 5 minutes).

Hint 1

Hint 2

Hint 3

Hint 4

Solution

ML Python

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Sample data placeholders (replace with actual data)
X_train, y_train = ...  # training features and labels
X_val, y_val = ...      # validation features and labels

model = Sequential([
    Dense(64, activation='relu', input_shape=(X_train.shape[1],)),
    Dropout(0.3),
    Dense(32, activation='relu'),
    Dropout(0.3),
    Dense(1, activation='sigmoid')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
              loss='binary_crossentropy',
              metrics=['accuracy'])

early_stop = EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

history = model.fit(X_train, y_train,
                    epochs=50,
                    batch_size=32,
                    validation_data=(X_val, y_val),
                    callbacks=[early_stop],
                    verbose=0)

train_acc = history.history['accuracy'][-1] * 100
val_acc = history.history['val_accuracy'][-1] * 100
train_loss = history.history['loss'][-1]
val_loss = history.history['val_loss'][-1]

print(f'Training accuracy: {train_acc:.2f}%, Validation accuracy: {val_acc:.2f}%')
print(f'Training loss: {train_loss:.3f}, Validation loss: {val_loss:.3f}')

Added dropout layers with 30% rate after dense layers to reduce overfitting.

Reduced learning rate to 0.001 for smoother training.

Added early stopping to stop training when validation loss stops improving.

Reduced the size of the second dense layer from 64 to 32 units to simplify the model.

Results Interpretation

Before: Training accuracy 90%, Validation accuracy 70%, Training loss 0.25, Validation loss 0.65

After: Training accuracy 86.5%, Validation accuracy 81.2%, Training loss 0.32, Validation loss 0.45

Adding dropout and early stopping helped reduce overfitting. The model now generalizes better, improving validation accuracy and making recommendations more reliable to drive user engagement.

Bonus Experiment

Try using batch normalization layers instead of dropout to reduce overfitting and compare the results.

💡 Hint

Batch normalization can stabilize and speed up training, sometimes improving generalization without dropping units.

Practice

1. Why do recommendation systems increase user engagement on websites or apps?

easy

A. They show users content or products that match their interests

B. They randomly display items without any user data

C. They reduce the number of items users can see

D. They force users to see ads only

Solution

1. Step 1: Understand user engagement

   User engagement means users spend more time and interact more with the platform.

2. Step 2: Role of recommendations

   Recommendations show users items they like, making them more interested and active.

3. Final Answer:

   They show users content or products that match their interests -> Option A

4. Quick Check:

   Relevant content = higher engagement [OK]

Hint: Recommendations match user interests to boost engagement [OK]

Common Mistakes:

• Thinking recommendations show random items
• Believing recommendations limit user choices
• Confusing ads with recommendations

2. Which of the following is the correct way to describe a recommendation system in simple terms?

easy

A. A system that shows the same items to all users

B. A system that guesses what users might like based on their past behavior

C. A system that deletes user data after each session

D. A system that only shows the newest items regardless of user interest

Solution

1. Step 1: Define recommendation system

   It uses past user data to predict what they might like next.

2. Step 2: Check options

   Only the system that guesses what users might like based on their past behavior describes this behavior correctly; others ignore personalization.

3. Final Answer:

   A system that guesses what users might like based on their past behavior -> Option B

4. Quick Check:

   Personalization = guessing user likes [OK]

Hint: Recommendations use past data to guess user preferences [OK]

Common Mistakes:

• Thinking recommendations show same items to everyone
• Confusing recommendation with data deletion
• Ignoring user history in recommendations

3. Consider this Python code snippet for a simple recommendation score calculation:

user_preferences = {'sports': 5, 'music': 3, 'movies': 4}
item_features = {'sports': 1, 'music': 0, 'movies': 1}
score = sum(user_preferences[cat] * item_features[cat] for cat in item_features)
print(score)

What will be the output of this code?

medium

A. 8

B. 10

C. 7

D. 9

Solution

1. Step 1: Calculate weighted scores

   Multiply user preference by item feature for each category: sports=5*1=5, music=3*0=0, movies=4*1=4.

2. Step 2: Sum the results

   Sum = 5 + 0 + 4 = 9.

3. Final Answer:

   9 -> Option D

4. Quick Check:

   5+0+4 = 9 [OK]

Hint: Multiply and sum matching categories for score [OK]

Common Mistakes:

• Adding all preference values without weights
• Ignoring zero feature values
• Multiplying wrong pairs of categories

4. The following code is intended to calculate a recommendation score but has a bug:

user_prefs = {'books': 4, 'games': 2}
item_feats = {'books': 1, 'games': 1}
score = 0
for cat in user_prefs:
    score += user_prefs[cat] * item_feats[cat]
print(score)

What is the error and how to fix it?

medium

A. KeyError if item_feats lacks a category; fix by iterating item_feats keys

B. SyntaxError due to missing colon; fix by adding colon after for loop

C. TypeError because score is int; fix by converting score to float

D. No error; code runs correctly

Solution

1. Step 1: Identify potential KeyError

   The loop uses keys from user_prefs but accesses item_feats; if item_feats misses a key, error occurs.

2. Step 2: Fix iteration to avoid missing keys

   Iterate over keys in item_feats or check key existence before access.

3. Final Answer:

   KeyError if item_feats lacks a category; fix by iterating item_feats keys -> Option A

4. Quick Check:

   Key mismatch causes error [OK]

Hint: Check keys exist before accessing dict values [OK]

Common Mistakes:

• Assuming all keys match in both dicts
• Ignoring possible KeyError exceptions
• Confusing syntax errors with runtime errors

5. A streaming service wants to improve engagement by recommending movies. They have user watch history and movie genres. Which approach best uses machine learning to drive engagement?

hard

A. Display only movies with the highest global ratings ignoring user data

B. Recommend only the newest movies regardless of user preferences

C. Build a model that predicts user ratings for unseen movies based on past watched genres

D. Show random movies from the entire catalog to all users

Solution

1. Step 1: Understand personalized recommendations

   Using user watch history and genres helps predict what each user likes.

2. Step 2: Evaluate options

   Build a model that predicts user ratings for unseen movies based on past watched genres uses machine learning to personalize, increasing engagement; others ignore personalization.

3. Final Answer:

   Build a model that predicts user ratings for unseen movies based on past watched genres -> Option C

4. Quick Check:

   Personalized prediction = better engagement [OK]

Hint: Use user history to predict preferences for recommendations [OK]

Common Mistakes:

• Ignoring user preferences in recommendations
• Assuming newest or highest rated always engage users
• Confusing random recommendations with personalized ones