Retraining strategies in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding when to retrain a model

You have a machine learning model deployed for predicting house prices. Over time, the model's accuracy drops significantly. What is the most likely reason you need to retrain the model?

A The model was trained with too many features initially.

B The model's code has a syntax error causing wrong predictions.

C The data distribution has changed, so the model no longer fits new data well.

D The model's training data was too large, causing slow predictions.

💡 Hint

Think about what happens when the world changes but the model stays the same.

When the data changes over time (called data drift), the model's predictions become less accurate. Retraining with new data helps the model adapt.

❓ Predict Output

intermediate

Output of incremental retraining code

What will be the output of the following Python code snippet that simulates incremental retraining?

ML Python

class Model:
    def __init__(self):
        self.data_count = 0
    def train(self, new_data):
        self.data_count += len(new_data)
    def predict(self):
        return self.data_count

model = Model()
model.train([1, 2, 3])
model.train([4, 5])
print(model.predict())

A None

B 5

C 2

D 3

💡 Hint

Count how many items were added in total.

The model trains twice, first with 3 items, then with 2 items. The total count is 5, which is returned by predict.

❓ Model Choice

advanced

Choosing a retraining strategy for streaming data

You have a model that receives continuous streaming data. Which retraining strategy is best to keep the model updated without retraining from scratch every time?

A Incremental learning that updates the model with new data only.

B Batch retraining using all historical data every day.

C Training a new model from scratch every hour.

D Freezing the model and never retraining.

💡 Hint

Think about efficiency and keeping the model fresh with new data.

Incremental learning updates the model with new data without full retraining, making it efficient for streaming data.

❓ Metrics

advanced

Evaluating retraining effectiveness

After retraining a classification model, which metric change best indicates the retraining improved the model?

A Accuracy increased and loss decreased on validation data.

B Training loss increased but validation accuracy increased.

C Validation loss increased and accuracy stayed the same.

D Accuracy decreased but training loss decreased.

💡 Hint

Look for improvements on data the model hasn't seen before.

Improved accuracy and lower loss on validation data show the model generalizes better after retraining.

🔧 Debug

expert

Debugging retraining code with data leakage

Consider this retraining code snippet. What is the main issue causing overly optimistic validation results?

ML Python

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X = [[i] for i in range(100)]
y = [0]*50 + [1]*50

# Split data
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

# Retrain model
model = LogisticRegression()
model.fit(X_train + X_val, y_train + y_val)

# Evaluate
score = model.score(X_val, y_val)
print(f"Validation accuracy: {score}")

A The random_state is fixed, causing biased splits.

B The test_size parameter is too large, reducing training data.

C The model is not fitted before scoring.

D The model is trained on validation data, causing data leakage.

💡 Hint

Check if validation data is used during training.

The code trains the model on both training and validation data, so validation accuracy is not a true measure of performance.

Practice

1. What is the main purpose of retraining a machine learning model?

easy

A. To remove features from the model permanently

B. To make the model run faster on old data

C. To reduce the size of the training dataset

D. To update the model with new data and keep it accurate

Solution

1. Step 1: Understand retraining purpose

   Retraining updates the model using new data to maintain or improve accuracy.

2. Step 2: Compare options

   Options B, C, and D do not describe retraining goals; only A matches the purpose.

3. Final Answer:

   To update the model with new data and keep it accurate -> Option D

4. Quick Check:

   Retraining = update model with new data [OK]

Hint: Retraining means learning from new data to stay accurate [OK]

Common Mistakes:

• Thinking retraining speeds up old data processing
• Confusing retraining with data reduction
• Believing retraining removes features

2. Which of the following is the correct way to fine-tune a pretrained model in Python using TensorFlow?

easy

A. model.fit(new_data, epochs=5, initial_epoch=0)

B. model.train(new_data, epochs=5)

C. model.evaluate(new_data, epochs=5)

D. model.fit(new_data, epochs=5, initial_epoch=existing_epochs)

Solution

1. Step 1: Understand fine-tuning syntax

   Fine-tuning a pretrained model typically uses model.fit starting from epoch 0 after loading pretrained weights.

2. Step 2: Analyze options

   A uses non-existent method 'train'. C uses evaluate, which doesn't train and lacks epochs param. D risks initial_epoch >= epochs error. B correctly trains with model.fit from epoch 0.

3. Final Answer:

   model.fit(new_data, epochs=5, initial_epoch=0) -> Option A

4. Quick Check:

   Fine-tune = model.fit(new_data, epochs=N, initial_epoch=0) [OK]

Hint: Fine-tune pretrained with model.fit(new_data, epochs=N) [OK]

Common Mistakes:

• Using model.train instead of model.fit
• Setting initial_epoch=existing_epochs causing error
• Confusing evaluation with training

3. Consider this Python code snippet for incremental retraining:

old_accuracy = 0.85
model.fit(new_data, epochs=3)
new_accuracy = 0.88
improvement = new_accuracy - old_accuracy
print(improvement)

What will be the printed output?

medium

A. 0.85

B. 0.88

C. 0.03

D. 3

Solution

1. Step 1: Calculate improvement

   Improvement is new_accuracy minus old_accuracy: 0.88 - 0.85 = 0.03.

2. Step 2: Understand print output

   The print statement outputs the improvement variable, which is 0.03.

3. Final Answer:

   0.03 -> Option C

4. Quick Check:

   0.88 - 0.85 = 0.03 [OK]

Hint: Subtract old accuracy from new to find improvement [OK]

Common Mistakes:

• Printing new_accuracy instead of improvement
• Confusing epochs with accuracy values
• Printing old_accuracy by mistake

4. You have this code snippet for retraining a model incrementally:

model.fit(new_data, epochs=5)
model.fit(new_data, epochs=3, initial_epoch=5)

But the second fit call throws an error. What is the likely cause?

medium

A. initial_epoch should be less than epochs

B. initial_epoch cannot be greater than or equal to epochs

C. You must reset the model before second fit

D. epochs must be the same in both fit calls

Solution

1. Step 1: Understand initial_epoch and epochs relation

   initial_epoch must be less than epochs in the current fit call to define the training range properly.

2. Step 2: Analyze the error cause

   Here, initial_epoch=5 is greater than epochs=3, which is invalid and causes an error.

3. Final Answer:

   initial_epoch cannot be greater than or equal to epochs -> Option B

4. Quick Check:

   initial_epoch < epochs required [OK]

Hint: initial_epoch must be less than epochs in fit call [OK]

Common Mistakes:

• Thinking model reset is needed
• Assuming epochs must match both calls
• Setting initial_epoch greater or equal to epochs

5. You have a model trained on data from 2020. New data from 2023 shows some feature distributions changed. Which retraining strategy best balances training time and model accuracy?

hard

A. Fine-tuning the existing model on new data with a low learning rate

B. Full retraining from scratch with combined old and new data

C. Incremental retraining only on new data without old data

D. No retraining; use the old model as is

Solution

1. Step 1: Consider data changes and retraining options

   Feature distribution changed, so model needs updating but full retraining is costly.

2. Step 2: Evaluate strategies

   Full retraining (A) is accurate but slow. Incremental retraining (C) risks forgetting old data. No retraining (D) ignores changes. Fine-tuning (B) updates model efficiently with new data.

3. Final Answer:

   Fine-tuning the existing model on new data with a low learning rate -> Option A

4. Quick Check:

   Fine-tune balances accuracy and training time [OK]

Hint: Fine-tune to update model efficiently with new data [OK]

Common Mistakes:

• Choosing full retraining despite high cost
• Ignoring old data in incremental retraining
• Skipping retraining when data changes