Loading model state_dict in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of loading a state_dict with missing keys?

Consider a PyTorch model and a saved state_dict that lacks some keys present in the model. What happens when you load this state_dict with strict=True?

PyTorch

import torch
import torch.nn as nn

class SimpleModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 5)
        self.fc2 = nn.Linear(5, 2)

model = SimpleModel()
saved_state = {'fc1.weight': torch.randn(5, 10), 'fc1.bias': torch.randn(5)}

try:
    model.load_state_dict(saved_state, strict=True)
    print('Loaded successfully')
except Exception as e:
    print(type(e).__name__)

A Loaded successfully

B KeyError

C RuntimeError

D TypeError

💡 Hint

Think about what strict=True means when keys are missing.

When loading a state_dict with strict=True, PyTorch expects all keys in the model to be present in the state_dict. Missing keys cause a RuntimeError.

❓ Model Choice

intermediate

Which option correctly loads a saved model state_dict ignoring missing keys?

You have a saved state_dict missing some keys. Which code snippet correctly loads it without error, ignoring missing keys?

A model.load_state_dict(saved_state, strict=False)

B model.load_state_dict(saved_state, strict=True)

C model.load_state_dict(saved_state, ignore_missing=True)

D model.load_state_dict(saved_state)

💡 Hint

Check the parameter that controls strict key matching.

Using strict=False allows loading a state_dict even if some keys are missing or extra, avoiding errors.

🔧 Debug

advanced

Why does this code raise a RuntimeError when loading a state_dict?

Examine the code below. Why does it raise a RuntimeError when loading the state_dict?

PyTorch

import torch
import torch.nn as nn

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.layer = nn.Linear(3, 2)

model = Model()
saved_state = {'layer.weight': torch.randn(2, 3), 'layer.bias': torch.randn(2), 'extra.weight': torch.randn(1)}

model.load_state_dict(saved_state)

A Because 'layer.weight' has wrong shape

B Because the model is not on the correct device

C Because 'layer.bias' is missing

D Because 'extra.weight' is not a key in the model's state_dict

💡 Hint

Look at keys in saved_state vs model's keys.

The saved_state contains an extra key 'extra.weight' which is not in the model's state_dict. By default, load_state_dict with strict=True raises a RuntimeError for unexpected keys.

❓ Hyperparameter

advanced

What does the strict parameter control in load_state_dict?

In PyTorch's load_state_dict method, what is the effect of setting strict=False?

A It forces the model to load only if all keys match exactly

B It allows loading state_dicts with missing or extra keys without error

C It loads the state_dict but resets optimizer parameters

D It converts all tensors to CPU before loading

💡 Hint

Think about key matching tolerance.

strict=False lets you load a state_dict even if some keys are missing or extra, avoiding errors.

🧠 Conceptual

expert

What is the best practice to load a state_dict when model architecture has changed?

You have updated your model architecture by adding new layers. You want to load weights from a previous checkpoint that lacks these new layers. What is the best practice to load the old weights safely?

A Load the state_dict with strict=False and manually initialize new layers

B Load the state_dict with strict=True to ensure all keys match

C Manually remove new layers from the model before loading state_dict

D Overwrite the model's state_dict with the checkpoint without loading

💡 Hint

Consider how to handle missing keys safely.

Using strict=False allows loading existing weights while ignoring new layers. Then you can initialize new layers separately.