Model versioning in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Model versioning

Problem:You have trained a simple neural network model for classifying handwritten digits using the MNIST dataset. You want to save different versions of your model after training with different hyperparameters to compare their performance later.

Current Metrics:Training accuracy: 98%, Validation accuracy: 96%

Issue:Currently, you save the model without versioning, which makes it hard to track and compare different model versions.

Your Task

Implement a model versioning system by saving multiple versions of your TensorFlow model with clear version names. Ensure you can load and evaluate any saved version easily.

Use TensorFlow's model saving and loading functions.

Save at least two versions of the model with different training epochs.

Do not change the model architecture.

Hint 1

Hint 2

Hint 3

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.datasets import mnist
import os

# Load data
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

# Build model function
def build_model():
    model = models.Sequential([
        layers.Flatten(input_shape=(28, 28)),
        layers.Dense(128, activation='relu'),
        layers.Dense(10, activation='softmax')
    ])
    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['accuracy'])
    return model

# Train and save version 1
model_v1 = build_model()
model_v1.fit(x_train, y_train, epochs=5, validation_split=0.1, verbose=0)
version1_path = 'saved_models/model_v1'
os.makedirs(version1_path, exist_ok=True)
model_v1.save(version1_path)

# Train and save version 2 with more epochs
model_v2 = build_model()
model_v2.fit(x_train, y_train, epochs=10, validation_split=0.1, verbose=0)
version2_path = 'saved_models/model_v2'
os.makedirs(version2_path, exist_ok=True)
model_v2.save(version2_path)

# Load and evaluate both versions
loaded_v1 = tf.keras.models.load_model(version1_path)
loaded_v2 = tf.keras.models.load_model(version2_path)

loss_v1, acc_v1 = loaded_v1.evaluate(x_test, y_test, verbose=0)
loss_v2, acc_v2 = loaded_v2.evaluate(x_test, y_test, verbose=0)

print(f'Version 1 - Loss: {loss_v1:.4f}, Accuracy: {acc_v1:.4f}')
print(f'Version 2 - Loss: {loss_v2:.4f}, Accuracy: {acc_v2:.4f}')

Created a function to build the model to reuse architecture.

Trained two models with different epochs (5 and 10).

Saved each model in a separate folder named with version numbers.

Loaded saved models from disk to evaluate and compare performance.

Results Interpretation

Before: Only one model saved without version info, hard to track changes.

After: Two models saved as 'model_v1' and 'model_v2' with different training epochs. Version 2 shows better accuracy (~98.5%) than version 1 (~97.5%).

Saving models with clear version names helps track improvements and compare different training runs easily. This is essential for managing machine learning projects effectively.

Bonus Experiment

Try saving models with timestamps instead of version numbers and load the latest model automatically for evaluation.

💡 Hint

Use Python's datetime module to generate timestamp strings and sort saved model folders by timestamp to find the latest.

Practice

1.

What is the main purpose of model versioning in TensorFlow?

easy

A. To visualize model architecture in graphs

B. To speed up model training by using GPUs

C. To convert models into different programming languages

D. To save and manage different copies of a model safely

Solution

1. Step 1: Understand model versioning concept

   Model versioning means saving multiple copies of a model with clear labels or versions.

2. Step 2: Identify the main purpose

   This helps to keep track of changes, compare models, and deploy the right one safely.

3. Final Answer:

   To save and manage different copies of a model safely -> Option D

4. Quick Check:

   Model versioning = save/manage copies [OK]

Hint: Model versioning means saving different model copies safely [OK]

Common Mistakes:

• Confusing versioning with speeding up training
• Thinking versioning changes model code
• Mixing versioning with visualization

2.

Which of the following is the correct way to save a TensorFlow model with a version number?

model.save(_____)

easy

A. 'my_model/v1'

B. 'my_model_version'

C. 'model_save'

D. 'v1_model_save/'

Solution

1. Step 1: Check folder naming for versioning

   Using a folder name like 'my_model/v1' clearly indicates version 1 of the model.

2. Step 2: Confirm correct syntax for saving

   The model.save() function accepts a path string; 'my_model/v1' is a valid path with version info.

3. Final Answer:

   'my_model/v1' -> Option A

4. Quick Check:

   Save path with version folder = 'my_model/v1' [OK]

Hint: Use folder paths with version numbers like 'model/v1' [OK]

Common Mistakes:

• Not including version info in folder name
• Using invalid folder path syntax
• Confusing file name with folder path

3.

Given the code below, what will be printed?

import tensorflow as tf

model = tf.keras.Sequential([tf.keras.layers.Dense(1)])
model.save('models/v2')
loaded_model = tf.keras.models.load_model('models/v2')
print(type(loaded_model))

medium

A. <class 'tensorflow.python.keras.engine.sequential.Sequential'>

B. <class 'tensorflow.keras.Model'>

C. <class 'tensorflow.keras.Sequential'>

D. <class 'tensorflow.keras.layers.Dense'>

Solution

1. Step 1: Understand model saving and loading

   Saving a Sequential model and loading it returns the same model type, which is tf.keras.Sequential.

2. Step 2: Check the printed type

   The type(loaded_model) will show <class 'tensorflow.keras.Sequential'> indicating the loaded model is a Sequential model.

3. Final Answer:

   <class 'tensorflow.keras.Sequential'> -> Option C

4. Quick Check:

   Loaded model type = Sequential [OK]

Hint: Loaded model keeps original class type like Sequential [OK]

Common Mistakes:

• Confusing model class with layer class
• Expecting generic Model class instead of Sequential
• Misreading printed type format

4.

What is wrong with this code snippet for loading a specific model version?

loaded_model = tf.keras.models.load_model('my_model')
# But the saved models are in folders 'my_model/v1', 'my_model/v2'

medium

A. The code should use model.load() instead

B. It tries to load a folder without version info, causing an error

C. The model path is correct; no error will occur

D. The model must be saved as a .h5 file to load

Solution

1. Step 1: Check the folder structure for saved models

   Models are saved in versioned folders like 'my_model/v1' and 'my_model/v2', not directly in 'my_model'.

2. Step 2: Identify the loading path issue

   Loading from 'my_model' will fail because that folder does not contain a saved model directly.

3. Final Answer:

   It tries to load a folder without version info, causing an error -> Option B

4. Quick Check:

   Load path must include version folder [OK]

Hint: Load model from exact version folder path [OK]

Common Mistakes:

• Ignoring version folder in load path
• Using wrong load function name
• Assuming .h5 file is always needed

5.

You have saved three versions of a model: model/v1, model/v2, and model/v3. You want to load the latest version automatically in your code. Which approach is best?

hard

A. Write code to list folders and select the highest version number folder

B. Manually update the path to the latest version each time

C. Always load model/v1 because it is the first

D. Save all versions in one folder and load without version info

Solution

1. Step 1: Understand the need for automatic version selection

   Manually changing paths is error-prone; automatic selection is better for deployment.

2. Step 2: Use code to list and pick the highest version folder

   Listing folders like 'v1', 'v2', 'v3' and selecting the highest number ensures loading the latest model.

3. Final Answer:

   Write code to list folders and select the highest version number folder -> Option A

4. Quick Check:

   Auto-select latest version by folder listing [OK]

Hint: List version folders and pick highest number automatically [OK]

Common Mistakes:

• Hardcoding version path causing manual errors
• Loading oldest version by mistake
• Ignoring version folders and mixing models