Saving weights only in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Saving weights only

Problem:You have trained a neural network model on a dataset, but the model file size is large because it saves the entire model architecture and optimizer state. You want to save only the model weights to reduce storage size and load them later into the same model architecture.

Current Metrics:Training accuracy: 92%, Validation accuracy: 89%, Model file size: 25MB

Issue:Saving the full model results in a large file size, which is inefficient for storage and sharing. You want to save only the weights without losing the ability to restore the model later.

Your Task

Save only the model weights after training and then load these weights into the same model architecture to make predictions. Confirm that the loaded model produces the same validation accuracy.

You must keep the model architecture code unchanged.

You cannot save the entire model (architecture + weights + optimizer).

You must use TensorFlow's recommended methods for saving and loading weights.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models

# Define the model architecture
model = models.Sequential([
    layers.Dense(64, activation='relu', input_shape=(20,)),
    layers.Dense(10, activation='softmax')
])

# Compile the model
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Generate dummy data for training and validation
import numpy as np
X_train = np.random.random((1000, 20))
y_train = np.random.randint(0, 10, 1000)
X_val = np.random.random((200, 20))
y_val = np.random.randint(0, 10, 200)

# Train the model
model.fit(X_train, y_train, epochs=5, batch_size=32, validation_data=(X_val, y_val))

# Save only the weights
weights_file = 'model_weights.weights.h5'
model.save_weights(weights_file)

# Create a new model instance with the same architecture
new_model = models.Sequential([
    layers.Dense(64, activation='relu', input_shape=(20,)),
    layers.Dense(10, activation='softmax')
])

# Compile the new model
new_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Load the saved weights into the new model
new_model.load_weights(weights_file)

# Evaluate the new model on validation data
loss, accuracy = new_model.evaluate(X_val, y_val, verbose=0)
print(f'Validation accuracy after loading weights: {accuracy * 100:.2f}%')

Used model.save_weights() to save only the weights to a file.

Created a new model instance with the same architecture.

Used model.load_weights() to load the saved weights into the new model.

Evaluated the new model to confirm it matches the original validation accuracy.

Changed weights file name to end with '.weights.h5' to comply with TensorFlow's recommended naming convention for saving weights in HDF5 format.

Results Interpretation

Before saving weights only: Model file size was 25MB including architecture and optimizer state.

After saving weights only: Weights file size is significantly smaller (usually a few MBs), and validation accuracy remains at 89% after loading weights into the same architecture.

Saving only the model weights reduces storage size and allows you to restore the model later by defining the same architecture and loading the weights. This is useful when you want to share or store models efficiently without saving the entire model configuration.

Bonus Experiment

Try saving and loading weights using the TensorFlow checkpoint format instead of HDF5 format.

💡 Hint

Use model.save_weights('path', save_format='tf') to save in TensorFlow format and load_weights() similarly.

Practice

1. What does saving weights only in TensorFlow mean?

easy

A. Saving the entire model including architecture and optimizer

B. Saving just the learned parameters without the model structure

C. Saving only the model architecture without weights

D. Saving the training dataset along with the model

Solution

1. Step 1: Understand what weights represent

   Weights are the learned parameters of a model after training.

2. Step 2: Differentiate saving weights only vs full model

   Saving weights only means storing these parameters without saving the model's structure or optimizer state.

3. Final Answer:

   Saving just the learned parameters without the model structure -> Option B

4. Quick Check:

   Saving weights only = learned parameters only [OK]

Hint: Weights only means parameters, not architecture or optimizer [OK]

Common Mistakes:

• Confusing weights with full model saving
• Thinking architecture is saved with weights only
• Assuming dataset is saved with weights

2. Which of the following is the correct syntax to save only weights of a TensorFlow model named model to a file weights.h5?

easy

A. model.save_weights(weights='weights.h5')

B. model.save('weights.h5')

C. model.save_weights('weights.h5')

D. tf.save_weights(model, 'weights.h5')

Solution

1. Step 1: Recall TensorFlow method to save weights

   The method to save weights is model.save_weights(filepath).

2. Step 2: Check syntax correctness

   model.save_weights('weights.h5') uses correct method and argument format. model.save('weights.h5') saves full model, C is invalid TensorFlow syntax, A uses wrong argument name.

3. Final Answer:

   model.save_weights('weights.h5') -> Option C

4. Quick Check:

   Use model.save_weights(filepath) to save weights [OK]

Hint: Use model.save_weights('filename') to save weights only [OK]

Common Mistakes:

• Using model.save() instead of save_weights()
• Incorrect argument names or order
• Trying to use non-existent TensorFlow functions

3. Consider this code snippet:

import tensorflow as tf
model = tf.keras.Sequential([
  tf.keras.layers.Dense(5, input_shape=(3,))
])
model.save_weights('w.h5')

new_model = tf.keras.Sequential([
  tf.keras.layers.Dense(5, input_shape=(3,))
])
new_model.load_weights('w.h5')
print(new_model.get_weights()[0].shape)

What will be the printed output?

medium

A. (3, 5)

B. (5, 3)

C. (5,)

D. Error: weights shape mismatch

Solution

1. Step 1: Understand model architecture

   The model has one Dense layer with 5 units and input shape (3,), so weights shape is (input_dim, units) = (3, 5).

2. Step 2: Loading weights into identical architecture

   Since new_model has the same architecture, loading weights works and weights shape remains (3, 5).

3. Final Answer:

   (3, 5) -> Option A

4. Quick Check:

   Weights shape = (input_dim, units) = (3, 5) [OK]

Hint: Weights shape = (input features, output units) in Dense layers [OK]

Common Mistakes:

• Confusing input and output dimensions order
• Assuming error when architectures match
• Expecting bias shape instead of weights

4. You saved weights using model.save_weights('w.h5'). When loading weights with new_model.load_weights('w.h5'), you get an error about mismatched shapes. What is the most likely cause?

medium

A. TensorFlow does not support loading weights from .h5 files

B. The weights file is corrupted

C. You forgot to compile the new model before loading weights

D. The new model architecture is different from the original model

Solution

1. Step 1: Understand weight loading requirements

   Weights can only be loaded if the new model has the same architecture as the original model.

2. Step 2: Analyze error cause

   Mismatched shapes error usually means layer sizes differ between saved and new model.

3. Final Answer:

   The new model architecture is different from the original model -> Option D

4. Quick Check:

   Model architectures must match to load weights [OK]

Hint: Check model architectures match before loading weights [OK]

Common Mistakes:

• Assuming file corruption without checking architecture
• Thinking compile is needed before loading weights
• Believing .h5 files are unsupported for weights

5. You want to share your trained model's learned parameters with a friend, but not the full model code. Which approach is best to save and share only the weights, and what must your friend do to use them?

hard

A. Save weights with model.save_weights(); friend recreates model architecture then loads weights

B. Save full model with model.save(); friend loads full model directly

C. Save weights as a numpy file; friend loads weights without model code

D. Save model architecture as JSON; friend loads architecture and trains from scratch

Solution

1. Step 1: Identify saving weights only method

   Use model.save_weights() to save only learned parameters without model structure.

2. Step 2: Explain how friend uses weights

   Friend must recreate the exact model architecture in code, then load weights with load_weights().

3. Final Answer:

   Save weights with model.save_weights(); friend recreates model architecture then loads weights -> Option A

4. Quick Check:

   Save weights only + recreate model to load weights [OK]

Hint: Save weights only, friend rebuilds model then loads weights [OK]

Common Mistakes:

• Thinking weights can be loaded without model code
• Confusing full model save with weights only
• Assuming JSON architecture saves weights