Saving and loading models in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - Saving and loading models

Which metric matters for saving and loading models and WHY

When saving and loading models, the key metric is model performance consistency. This means the model should give the same predictions and accuracy after loading as it did before saving. We check metrics like accuracy, loss, or any task-specific score before saving and after loading to make sure the model was saved correctly without losing information.

Confusion matrix or equivalent visualization

Imagine a classification model saved and loaded. We compare the confusion matrix before saving and after loading:

    Before saving:
      TP=50  FP=10
      FN=5   TN=35

    After loading:
      TP=50  FP=10
      FN=5   TN=35
    

If these numbers match exactly, the model was saved and loaded correctly.

Precision vs Recall tradeoff with concrete examples

Saving and loading models does not directly affect precision or recall tradeoffs. But if the model changes after loading, these metrics can shift unexpectedly. For example, if a cancer detection model's recall drops after loading, it means the model lost some ability to find cancer cases. This shows the importance of verifying metrics after loading.

What "good" vs "bad" metric values look like for this use case

Good: The model's accuracy, precision, recall, and loss before saving and after loading are the same or very close (differences within a tiny margin like 0.001). This means the model was saved and loaded without damage.

Bad: Metrics change a lot after loading. For example, accuracy drops from 90% to 70%, or loss increases significantly. This means the saved model file is corrupted or incomplete.

Metrics pitfalls

• Not verifying metrics after loading: Assuming the model works without checking can hide problems.
• Data leakage: If the test data leaks into training before saving, metrics look good but are misleading.
• Overfitting indicators: If the model performs perfectly on training data but poorly on test data after loading, it may be overfitting.
• Version mismatch: Saving a model in one software version and loading in another can cause errors or metric changes.

Self-check question

Your model has 98% accuracy before saving but after loading, accuracy is 85%. Is it good for production? Why or why not?

Answer: No, it is not good. The large drop in accuracy means the model was not saved or loaded correctly. You must fix this before using the model in production to avoid wrong predictions.

Key Result

Model performance metrics must remain consistent before saving and after loading to ensure reliability.

Practice

1. What is the main purpose of saving a trained machine learning model?

easy

A. To increase the model's training speed

B. To reuse the model later without retraining

C. To improve the model's accuracy automatically

D. To delete the training data after use

Solution

1. Step 1: Understand model saving

   Saving a model stores its learned parameters so it can be reused later.

2. Step 2: Identify the main benefit

   This avoids retraining, saving time and resources.

3. Final Answer:

   To reuse the model later without retraining -> Option B

4. Quick Check:

   Saving = reuse without retraining [OK]

Hint: Saving means keep model for later use without retraining [OK]

Common Mistakes:

• Thinking saving improves accuracy automatically
• Confusing saving with speeding up training
• Believing saving deletes data

2. Which of the following is the correct way to save a Keras model named model to a file called my_model.h5?

easy

A. model.save('my_model.h5')

B. save_model('my_model.h5', model)

C. model.save_model('my_model.h5')

D. save('model', 'my_model.h5')

Solution

1. Step 1: Recall Keras save syntax

   Keras models are saved using model.save(filename).

2. Step 2: Match correct syntax

   model.save('my_model.h5') matches this syntax exactly.

3. Final Answer:

   model.save('my_model.h5') -> Option A

4. Quick Check:

   model.save(filename) saves model [OK]

Hint: Use model.save('filename') to save Keras models [OK]

Common Mistakes:

• Using incorrect function names like save_model or save
• Swapping argument order
• Calling save_model as a method on model

3. What will be the output of the following code snippet?

from tensorflow.keras.models import load_model
model = load_model('saved_model.h5')
predictions = model.predict([[5.1, 3.5, 1.4, 0.2]])
print(predictions)

medium

A. The model architecture printed as text

B. An error because load_model requires a directory, not a file

C. A list of class labels as strings

D. A numpy array of prediction probabilities

Solution

1. Step 1: Understand load_model usage

   load_model loads a saved Keras model from a file.

2. Step 2: Predict method output

   model.predict returns a numpy array of prediction probabilities for input data.

3. Final Answer:

   A numpy array of prediction probabilities -> Option D

4. Quick Check:

   load_model + predict = numpy array output [OK]

Hint: load_model loads model; predict returns probabilities array [OK]

Common Mistakes:

• Thinking load_model only loads from directories
• Expecting printed model architecture from predict
• Assuming predict returns class labels directly

4. You try to load a saved model using load_model('model.h5') but get an error: OSError: Unable to open file (file signature not found). What is the most likely cause?

medium

A. The model file is too large to load

B. You forgot to import load_model from keras

C. The file 'model.h5' is corrupted or not a valid model file

D. The model was saved with a different Python version

Solution

1. Step 1: Analyze the error message

   OSError about file signature means the file is not recognized as a valid HDF5 model file.

2. Step 2: Identify common causes

   This usually happens if the file is corrupted or not a saved model file.

3. Final Answer:

   The file 'model.h5' is corrupted or not a valid model file -> Option C

4. Quick Check:

   File signature error = corrupted or invalid file [OK]

Hint: File signature errors mean corrupted or wrong file type [OK]

Common Mistakes:

• Assuming Python version mismatch causes this error
• Forgetting to import but error message differs
• Thinking file size causes signature errors

5. You have a trained model saved as model_v1.h5. You want to load it, continue training on new data, then save the updated model as model_v2.h5. Which code snippet correctly does this?

hard

A. from tensorflow.keras.models import load_model
model = load_model('model_v1.h5')
model.fit(new_data, new_labels, epochs=5)
model.save('model_v2.h5')

B. model = load('model_v1.h5')
model.train(new_data, new_labels)
save_model(model, 'model_v2.h5')

C. load_model('model_v1.h5')
train_model(new_data, new_labels)
save('model_v2.h5')

D. from keras import load
model = load('model_v1.h5')
model.fit(new_data, new_labels)
model.save('model_v2.h5')

Solution

1. Step 1: Load the saved model correctly

   Use load_model('model_v1.h5') from tensorflow.keras.models to load the model.

2. Step 2: Continue training and save updated model

   Call model.fit() with new data, then save with model.save('model_v2.h5').

3. Final Answer:

   from tensorflow.keras.models import load_model model = load_model('model_v1.h5') model.fit(new_data, new_labels, epochs=5) model.save('model_v2.h5') -> Option A

4. Quick Check:

   Load, fit, save = from tensorflow.keras.models import load_model model = load_model('model_v1.h5') model.fit(new_data, new_labels, epochs=5) model.save('model_v2.h5') [OK]

Hint: Load with load_model, fit new data, then save again [OK]

Common Mistakes:

• Using undefined functions like load or save_model
• Forgetting to import load_model
• Not calling fit before saving updated model