Loading model state_dict in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Loading model state_dict

Which metric matters for Loading model state_dict and WHY

When loading a model's state_dict, the key metric is model accuracy or performance metrics after loading. This is because loading the weights correctly should restore the model's learned knowledge. If accuracy or loss after loading matches the saved model's performance, the loading was successful.

Metrics like loss, accuracy, precision, or recall measured on a validation set after loading confirm the model state was restored properly.

Confusion matrix or equivalent visualization

After loading the state_dict, you can evaluate the model on a test set and get a confusion matrix like this:

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP)  | False Negative (FN) |
      | False Positive (FP) | True Negative (TN)  |
    

For example, if the model was a classifier, the confusion matrix shows how well the loaded model predicts classes. If the matrix matches the saved model's results, loading was correct.

Precision vs Recall tradeoff with concrete examples

Loading the state_dict itself does not affect precision or recall directly, but if loading is faulty, these metrics will drop.

For example, if a cancer detection model is loaded incorrectly, recall (catching all cancer cases) may drop, which is bad. If precision drops, the model may give many false alarms.

So, after loading, check precision and recall to ensure the model behaves as expected.

What "good" vs "bad" metric values look like for this use case

Good: After loading, the model's accuracy, precision, recall, and loss are close to the values before saving. For example, accuracy remains above 90%, loss stays low, and confusion matrix values are consistent.

Bad: After loading, accuracy drops significantly (e.g., from 90% to 50%), loss increases, or confusion matrix shows many misclassifications. This means the state_dict was not loaded correctly or is corrupted.

Metrics pitfalls

• Mismatch in model architecture: Loading a state_dict into a different model structure causes errors or wrong weights, leading to poor metrics.
• Partial loading: Loading only some layers' weights can cause unexpected performance drops.
• Data leakage: Evaluating on training data after loading can give misleadingly high accuracy.
• Overfitting indicators: If metrics after loading are perfect on training but poor on validation, the model may be overfitted.

Self-check question

Your model has 98% accuracy but 12% recall on fraud detection after loading the state_dict. Is it good for production? Why or why not?

Answer: No, it is not good. High accuracy can be misleading if the dataset is imbalanced (few fraud cases). The very low recall (12%) means the model misses most fraud cases, which is critical in fraud detection. You want high recall to catch as many frauds as possible.

Key Result

After loading a model's state_dict, matching pre-save accuracy and recall confirms correct restoration of learned knowledge.

Practice

1. What does model.load_state_dict() do in PyTorch?

easy

A. It loads saved model weights into the model.

B. It saves the current model weights to a file.

C. It initializes a new model architecture.

D. It compiles the model for training.

Solution

1. Step 1: Understand the purpose of load_state_dict

   This function is used to load previously saved weights into a model.

2. Step 2: Differentiate from other functions

   Saving weights uses state_dict() with torch.save(), not load_state_dict().

3. Final Answer:

   It loads saved model weights into the model. -> Option A

4. Quick Check:

   Load weights = load_state_dict() [OK]

Hint: Remember: load_state_dict loads weights, not saves them [OK]

Common Mistakes:

• Confusing loading weights with saving weights
• Thinking it initializes model architecture
• Assuming it compiles the model

2. Which of the following is the correct syntax to load a saved state dictionary from a file model.pth into a model named model?

easy

A. model.load_state_dict(torch.load('model.pth'))

B. model.load(torch.load_state_dict('model.pth'))

C. torch.load_state_dict(model, 'model.pth')

D. model.load_state_dict('model.pth')

Solution

1. Step 1: Identify correct function usage

   The correct way is to first load the saved weights with torch.load() and then pass them to model.load_state_dict().

2. Step 2: Check syntax correctness

   model.load_state_dict(torch.load('model.pth')) correctly calls torch.load('model.pth') inside model.load_state_dict(). Other options misuse function names or argument order.

3. Final Answer:

   model.load_state_dict(torch.load('model.pth')) -> Option A

4. Quick Check:

   Load weights with torch.load, then load_state_dict [OK]

Hint: Load file with torch.load, then pass to load_state_dict [OK]

Common Mistakes:

• Passing filename directly to load_state_dict
• Using wrong function names or order
• Confusing torch.load and load_state_dict

3. Given the code below, what will be printed?

import torch
import torch.nn as nn

class SimpleModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(2, 1)

model = SimpleModel()
torch.save(model.state_dict(), 'temp.pth')

new_model = SimpleModel()
new_model.load_state_dict(torch.load('temp.pth'))

print(all(torch.equal(p1, p2) for p1, p2 in zip(model.parameters(), new_model.parameters())))

medium

A. Raises an error

B. False

C. True

D. None

Solution

1. Step 1: Understand saving and loading state_dict

   The code saves the original model's weights and loads them into a new model instance.

2. Step 2: Compare parameters of both models

   Since the new model loads the exact saved weights, parameters should be identical, so the comparison returns True.

3. Final Answer:

   True -> Option C

4. Quick Check:

   Loaded weights match saved weights = True [OK]

Hint: Loaded model matches saved weights exactly [OK]

Common Mistakes:

• Assuming new model has random weights after loading
• Thinking load_state_dict changes model architecture
• Expecting an error due to missing device argument

4. You try to load a saved state_dict into your model but get this error: RuntimeError: Error(s) in loading state_dict for Model: Missing key(s) in state_dict: "fc.weight". What is the most likely cause?

medium

A. The file path to the saved state_dict is incorrect.

B. The saved state_dict is from a different model architecture.

C. You forgot to call torch.load() before loading.

D. The model was not moved to the correct device before loading.

Solution

1. Step 1: Analyze the error message

   The error says some keys are missing in the loaded state_dict, meaning the model expects parameters not found in the saved weights.

2. Step 2: Identify cause of missing keys

   This usually happens when the saved weights come from a different model architecture than the current model.

3. Final Answer:

   The saved state_dict is from a different model architecture. -> Option B

4. Quick Check:

   Missing keys = architecture mismatch [OK]

Hint: Missing keys usually mean model architectures differ [OK]

Common Mistakes:

• Assuming file path error causes missing keys
• Forgetting to load file before loading state_dict
• Thinking device mismatch causes missing keys

5. You have a model trained on GPU and saved its state_dict. Now you want to load it on a CPU-only machine. Which code snippet correctly loads the weights without error?

hard

A. model.load_state_dict(torch.load('model_gpu.pth', device='cpu'))

B. model.load_state_dict(torch.load('model_gpu.pth'))

C. model.load_state_dict(torch.load('model_gpu.pth', map_location='cuda'))

D. model.load_state_dict(torch.load('model_gpu.pth', map_location=torch.device('cpu')))

Solution

1. Step 1: Understand device mismatch issue

   Loading GPU-trained weights on CPU requires mapping the storage to CPU to avoid errors.

2. Step 2: Use correct map_location argument

   Passing map_location=torch.device('cpu') to torch.load() correctly maps tensors to CPU.

3. Final Answer:

   model.load_state_dict(torch.load('model_gpu.pth', map_location=torch.device('cpu'))) -> Option D

4. Quick Check:

   Use map_location to load GPU weights on CPU [OK]

Hint: Use map_location=torch.device('cpu') when loading GPU weights on CPU [OK]

Common Mistakes:

• Not using map_location causes runtime errors
• Passing wrong device string like 'cuda' on CPU
• Using non-existent 'device' argument in torch.load