Model versioning in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Model versioning

Problem:You have trained a simple neural network model for classifying handwritten digits using the MNIST dataset. You want to save different versions of your model after training with different hyperparameters to compare their performance later.

Current Metrics:Training accuracy: 98%, Validation accuracy: 96%

Issue:Currently, you save the model without versioning, which makes it hard to track and compare different model versions.

Your Task

Implement a model versioning system by saving multiple versions of your TensorFlow model with clear version names. Ensure you can load and evaluate any saved version easily.

Use TensorFlow's model saving and loading functions.

Save at least two versions of the model with different training epochs.

Do not change the model architecture.

Hint 1

Hint 2

Hint 3

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.datasets import mnist
import os

# Load data
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

# Build model function
def build_model():
    model = models.Sequential([
        layers.Flatten(input_shape=(28, 28)),
        layers.Dense(128, activation='relu'),
        layers.Dense(10, activation='softmax')
    ])
    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['accuracy'])
    return model

# Train and save version 1
model_v1 = build_model()
model_v1.fit(x_train, y_train, epochs=5, validation_split=0.1, verbose=0)
version1_path = 'saved_models/model_v1'
os.makedirs(version1_path, exist_ok=True)
model_v1.save(version1_path)

# Train and save version 2 with more epochs
model_v2 = build_model()
model_v2.fit(x_train, y_train, epochs=10, validation_split=0.1, verbose=0)
version2_path = 'saved_models/model_v2'
os.makedirs(version2_path, exist_ok=True)
model_v2.save(version2_path)

# Load and evaluate both versions
loaded_v1 = tf.keras.models.load_model(version1_path)
loaded_v2 = tf.keras.models.load_model(version2_path)

loss_v1, acc_v1 = loaded_v1.evaluate(x_test, y_test, verbose=0)
loss_v2, acc_v2 = loaded_v2.evaluate(x_test, y_test, verbose=0)

print(f'Version 1 - Loss: {loss_v1:.4f}, Accuracy: {acc_v1:.4f}')
print(f'Version 2 - Loss: {loss_v2:.4f}, Accuracy: {acc_v2:.4f}')

Created a function to build the model to reuse architecture.

Trained two models with different epochs (5 and 10).

Saved each model in a separate folder named with version numbers.

Loaded saved models from disk to evaluate and compare performance.

Results Interpretation

Before: Only one model saved without version info, hard to track changes.

After: Two models saved as 'model_v1' and 'model_v2' with different training epochs. Version 2 shows better accuracy (~98.5%) than version 1 (~97.5%).

Saving models with clear version names helps track improvements and compare different training runs easily. This is essential for managing machine learning projects effectively.

Bonus Experiment

Try saving models with timestamps instead of version numbers and load the latest model automatically for evaluation.

💡 Hint

Use Python's datetime module to generate timestamp strings and sort saved model folders by timestamp to find the latest.