Loading and inference in TensorFlow

Introduction

Loading a saved model lets you reuse it without retraining. Inference means using the model to make predictions on new data.

You trained a model and want to use it later without retraining.

You want to predict if an email is spam or not using a saved model.

You have a saved image classifier and want to identify new pictures.

You want to deploy a model in an app to make real-time predictions.

Syntax

TensorFlow

import tensorflow as tf

# Load the saved model
model = tf.keras.models.load_model('path_to_model')

# Prepare input data (example: numpy array)
input_data = ...

# Make prediction
predictions = model.predict(input_data)

Use tf.keras.models.load_model() to load a saved model.

Input data shape must match what the model expects.

Examples

Load a model and predict on one sample with 4 features.

TensorFlow

import tensorflow as tf

model = tf.keras.models.load_model('my_model')

import numpy as np
input_data = np.array([[5.1, 3.5, 1.4, 0.2]])
predictions = model.predict(input_data)
print(predictions)

Predict on a batch of two samples at once.

TensorFlow

import tensorflow as tf
model = tf.keras.models.load_model('saved_model')

import numpy as np
batch_data = np.array([[5.1, 3.5, 1.4, 0.2], [6.2, 3.4, 5.4, 2.3]])
preds = model.predict(batch_data)
print(preds)

Sample Model

This program trains a small model on dummy data, saves it, loads it back, and makes a prediction on new data.

TensorFlow

import tensorflow as tf
import numpy as np

# Create and train a simple model
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

model = Sequential([
    Dense(10, activation='relu', input_shape=(4,)),
    Dense(3, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Dummy data: 5 samples, 4 features each
x_train = np.array([
    [5.1, 3.5, 1.4, 0.2],
    [6.2, 3.4, 5.4, 2.3],
    [5.9, 3.0, 4.2, 1.5],
    [6.9, 3.1, 5.1, 2.3],
    [5.5, 2.3, 4.0, 1.3]
])

# Labels for 3 classes
y_train = np.array([0, 2, 1, 2, 1])

# Train briefly
model.fit(x_train, y_train, epochs=3, verbose=1)

# Save the model
model.save('my_simple_model')

# Load the model
loaded_model = tf.keras.models.load_model('my_simple_model')

# New data for inference
x_new = np.array([[6.0, 3.0, 4.8, 1.8]])

# Predict
predictions = loaded_model.predict(x_new)
print('Predictions:', predictions)

# Show predicted class
predicted_class = predictions.argmax(axis=1)
print('Predicted class:', predicted_class[0])

Important Notes

Make sure the input data shape matches the model's input shape exactly.

Loading a model restores its architecture, weights, and optimizer state.

Inference is usually done with model.predict() which returns probabilities or outputs.

Summary

Load saved models with tf.keras.models.load_model().

Use model.predict() to get predictions on new data.

Ensure input data shape matches what the model expects.

Practice

1. What is the main purpose of tf.keras.models.load_model() in TensorFlow?

easy

A. To visualize the model architecture

B. To load a previously saved model for inference or further training

C. To compile a model with loss and optimizer

D. To create a new model architecture from scratch

Solution

1. Step 1: Understand the function purpose

   tf.keras.models.load_model() is used to load a model that was saved earlier, including its architecture, weights, and optimizer state.

2. Step 2: Differentiate from other functions

   Creating a new model or compiling it are separate steps. Loading is specifically for retrieving a saved model.

3. Final Answer:

   To load a previously saved model for inference or further training -> Option B

4. Quick Check:

   Load saved model = load_model() [OK]

Hint: Load saved models with load_model() function [OK]

Common Mistakes:

• Confusing load_model() with model creation
• Thinking load_model() compiles the model
• Assuming load_model() only loads weights

2. Which of the following is the correct way to get predictions from a loaded TensorFlow model named model on input data new_data?

easy

A. predictions = model.predict(new_data)

B. predictions = model.fit(new_data)

C. predictions = model.evaluate(new_data)

D. predictions = model.compile(new_data)

Solution

1. Step 1: Identify the correct method for inference

   To get predictions from a model, the method predict() is used.

2. Step 2: Understand other methods

   fit() trains the model, evaluate() tests performance, and compile() sets up the model.

3. Final Answer:

   predictions = model.predict(new_data) -> Option A

4. Quick Check:

   Predict new data = model.predict() [OK]

Hint: Use model.predict() to get predictions [OK]

Common Mistakes:

• Using fit() instead of predict() for inference
• Calling evaluate() expecting predictions
• Trying to compile() during prediction

3. Given the following code, what will be the shape of predictions?

import tensorflow as tf
import numpy as np
model = tf.keras.Sequential([
  tf.keras.layers.Dense(3, input_shape=(4,))
])
model.compile(optimizer='adam', loss='mse')
input_data = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
predictions = model.predict(input_data)

medium

A. (3, 4)

B. (4, 3)

C. (2, 3)

D. (2, 4)

Solution

1. Step 1: Analyze input shape and batch size

   The input data has shape (2, 4) meaning 2 samples, each with 4 features.

2. Step 2: Determine output shape from model layer

   The Dense layer outputs 3 units per sample, so output shape is (2, 3).

3. Final Answer:

   (2, 3) -> Option C

4. Quick Check:

   Input batch 2, output units 3 = (2, 3) [OK]

Hint: Output shape = (batch_size, units in last layer) [OK]

Common Mistakes:

• Confusing input shape with output shape
• Mixing batch size and feature dimensions
• Assuming output shape matches input shape

4. You loaded a model with model = tf.keras.models.load_model('my_model') but get this error when calling model.predict(new_data): ValueError: Error when checking input: expected dense_input to have shape (4,) but got array with shape (3,). What is the likely cause?

medium

A. The input data shape does not match the model's expected input shape

B. The model was not compiled before prediction

C. The model file is corrupted

D. The predict() method was called without training the model

Solution

1. Step 1: Understand the error message

   The error says the model expects input shape (4,), but received (3,), meaning input data shape is wrong.

2. Step 2: Rule out other causes

   Model compilation is not required for prediction, corrupted files cause different errors, and training is not needed for predict().

3. Final Answer:

   The input data shape does not match the model's expected input shape -> Option A

4. Quick Check:

   Input shape mismatch = ValueError [OK]

Hint: Match input data shape to model input shape exactly [OK]

Common Mistakes:

• Ignoring input shape mismatch
• Thinking model must be compiled before predict
• Assuming predict requires training first

5. You have a saved TensorFlow model that expects input shape (None, 5). You want to predict on a single sample with 5 features stored in a list sample = [1, 2, 3, 4, 5]. Which code snippet correctly prepares the input and gets the prediction?

hard

A. input_data = np.array(sample)
prediction = model.predict(input_data)

B. input_data = np.array(sample).reshape(1, 1, 5)
prediction = model.predict(input_data)

C. input_data = np.array(sample).reshape(5, 1)
prediction = model.predict(input_data)

D. import numpy as np
input_data = np.array([sample])
prediction = model.predict(input_data)

Solution

1. Step 1: Understand model input shape

   The model expects input shape (None, 5), where None is batch size, so input must be 2D with shape (batch_size, 5).

2. Step 2: Prepare input data correctly

   Wrapping the sample list in another list creates shape (1, 5), which matches expected input.

3. Step 3: Check other options

   input_data = np.array(sample) prediction = model.predict(input_data) gives shape (5,), not 2D; input_data = np.array(sample).reshape(5, 1) prediction = model.predict(input_data) reshapes to (5,1) which is wrong; input_data = np.array(sample).reshape(1, 1, 5) prediction = model.predict(input_data) reshapes to (1,1,5) which is 3D and incorrect.

4. Final Answer:

   import numpy as np input_data = np.array([sample]) prediction = model.predict(input_data) -> Option D

5. Quick Check:

   Input shape (1,5) matches model input [OK]

Hint: Wrap single sample in list to add batch dimension [OK]

Common Mistakes:

• Passing 1D array instead of 2D batch input
• Incorrect reshaping causing wrong input dimensions
• Adding extra unnecessary dimensions