ONNX Runtime inference in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to load an ONNX model using ONNX Runtime.

PyTorch

import onnxruntime as ort

session = ort.InferenceSession([1])

Drag options to blanks, or click blank then click option'

A"model.onnx"

Bmodel.onnx

Cload_model("model.onnx")

Donnx.load("model.onnx")

Attempts:

3 left

💡 Hint

Common Mistakes

Passing the model filename without quotes causes a NameError.

Using onnx.load returns a model object, not a path string.

The InferenceSession constructor expects the path to the ONNX model as a string.

2fill in blank

medium

Complete the code to prepare input data as a dictionary for ONNX Runtime inference.

PyTorch

import numpy as np

input_name = session.get_inputs()[0].name
input_data = np.array([[1.0, 2.0, 3.0]], dtype=np.float32)
inputs = [1]

Drag options to blanks, or click blank then click option'

A{input_name: input_data}

B[input_name, input_data]

C(input_name, input_data)

Dinput_name + input_data

Attempts:

3 left

💡 Hint

Common Mistakes

Using a list or tuple instead of a dictionary causes runtime errors.

Concatenating strings and arrays is invalid.

ONNX Runtime expects inputs as a dictionary mapping input names to numpy arrays.

3fill in blank

hard

Fix the error in the code to run inference and get the output.

PyTorch

outputs = session.run(None, [1])
print(outputs[0])

Drag options to blanks, or click blank then click option'

Ainput_name

Binput_data

Cinputs

Dsession.get_inputs()

Attempts:

3 left

💡 Hint

Common Mistakes

Passing only the numpy array causes a TypeError.

Passing the input name string causes a runtime error.

The run method requires a dictionary of inputs, not just the data array or input name.

4fill in blank

hard

Fill both blanks to get the name of the first output and print its shape.

PyTorch

output_name = session.get_outputs()[[1]].name
print(session.run([output_name], [2])[0].shape)

Drag options to blanks, or click blank then click option'

A0

Binputs

C1

Dinput_data

Attempts:

3 left

💡 Hint

Common Mistakes

Using index 1 accesses a non-first output which may not exist.

Passing raw input data instead of dictionary causes errors.

The first output is at index 0, and the inputs dictionary must be passed to run.

5fill in blank

hard

Fill all three blanks to run inference on multiple inputs and get the second output.

PyTorch

input_names = [inp.name for inp in session.get_inputs()]
inputs = {name: np.random.rand(1, 3).astype(np.float32) for name in input_names}
outputs = session.run([session.get_outputs()[[1]].name], [2])
print(outputs[0].shape)

# Access input for first input name
first_input = inputs[[3]]

Drag options to blanks, or click blank then click option'

A1

Binputs

Cinput_names[0]

Dinput_names[1]

Attempts:

3 left

💡 Hint

Common Mistakes

Using wrong output index causes index errors.

Passing raw data instead of inputs dictionary causes errors.

Accessing inputs with wrong key causes KeyError.

The second output is at index 1, inputs dictionary is passed to run, and the first input name is input_names[0].

Practice

1. What is the main purpose of using ONNX Runtime with a PyTorch model?

easy

A. To train the model with more data

B. To convert the model into a TensorFlow format

C. To run the model faster and on different devices

D. To visualize the model architecture

Solution

1. Step 1: Understand ONNX Runtime's role

   ONNX Runtime is designed to run machine learning models efficiently across various hardware.

2. Step 2: Compare options

   Training or converting to TensorFlow is not the main purpose; visualization is unrelated.

3. Final Answer:

   To run the model faster and on different devices -> Option C

4. Quick Check:

   ONNX Runtime = fast, cross-device inference [OK]

Hint: ONNX Runtime is for fast, flexible model running [OK]

Common Mistakes:

• Confusing inference with training
• Thinking ONNX Runtime converts models to other frameworks
• Assuming it is for visualization

2. Which of the following is the correct way to load an ONNX model using ONNX Runtime in Python?

easy

A. session = onnxruntime.load_model('model.onnx')

B. session = onnx.load('model.onnx')

C. session = torch.load('model.onnx')

D. session = onnxruntime.InferenceSession('model.onnx')

Solution

1. Step 1: Recall ONNX Runtime loading syntax

   ONNX Runtime uses InferenceSession to load ONNX models.

2. Step 2: Check each option

   session = onnxruntime.InferenceSession('model.onnx') uses correct syntax. session = onnx.load('model.onnx') is for ONNX library, not runtime. session = torch.load('model.onnx') is PyTorch loading, not ONNX. session = onnxruntime.load_model('model.onnx') is invalid method.

3. Final Answer:

   session = onnxruntime.InferenceSession('model.onnx') -> Option D

4. Quick Check:

   InferenceSession loads ONNX models [OK]

Hint: Use InferenceSession to load ONNX models [OK]

Common Mistakes:

• Using torch.load for ONNX files
• Confusing onnx.load with onnxruntime
• Trying non-existent load_model method

3. Given the following code snippet, what will be the output of print(output)?

import onnxruntime
import numpy as np

session = onnxruntime.InferenceSession('model.onnx')
input_name = session.get_inputs()[0].name
input_data = np.array([[1.0, 2.0, 3.0]], dtype=np.float32)
output = session.run(None, {input_name: input_data})
print(output)

medium

A. A list containing the model's prediction array

B. A numpy array of input_data

C. An error because input_data is not a tensor

D. None, because session.run returns nothing

Solution

1. Step 1: Understand session.run output

   session.run returns a list of numpy arrays with model outputs.

2. Step 2: Analyze code behavior

   Input is correctly prepared as numpy array; no error occurs; output is a list with prediction arrays.

3. Final Answer:

   A list containing the model's prediction array -> Option A

4. Quick Check:

   session.run returns list of outputs [OK]

Hint: session.run returns list of numpy arrays [OK]

Common Mistakes:

• Expecting a single numpy array instead of list
• Thinking input must be a PyTorch tensor
• Assuming session.run returns None

4. You run ONNX Runtime inference but get an error: TypeError: expected np.ndarray (got list). What is the likely cause?

medium

A. Output is not captured correctly

B. Input data was passed as a Python list instead of a numpy array

C. ONNX Runtime session was not created

D. Model file is corrupted

Solution

1. Step 1: Interpret the error message

   The error says input expected is numpy array but got list.

2. Step 2: Identify cause

   Passing input as Python list instead of np.ndarray causes this error.

3. Final Answer:

   Input data was passed as a Python list instead of a numpy array -> Option B

4. Quick Check:

   Input must be numpy array, not list [OK]

Hint: Always convert inputs to numpy arrays before inference [OK]

Common Mistakes:

• Passing Python lists directly as input
• Assuming model file corruption causes this error
• Ignoring session creation errors

5. You want to run inference on a batch of 5 images using ONNX Runtime. Your model expects input shape (batch_size, 3, 224, 224). Which code snippet correctly prepares the input and runs inference?

hard

A. import numpy as np
batch_input = np.random.rand(5, 3, 224, 224).astype(np.float32)
output = session.run(None, {input_name: batch_input})

B. batch_input = torch.rand(5, 3, 224, 224)
output = session.run(None, {input_name: batch_input})

C. batch_input = np.random.rand(3, 224, 224).astype(np.float32)
output = session.run(None, {input_name: batch_input})

D. batch_input = np.random.rand(5, 224, 224, 3).astype(np.float32)
output = session.run(None, {input_name: batch_input})

Solution

1. Step 1: Check input shape requirements

   Model expects (batch_size, 3, 224, 224) so batch size 5 means shape (5,3,224,224).

2. Step 2: Validate each option's input shape and type

   import numpy as np batch_input = np.random.rand(5, 3, 224, 224).astype(np.float32) output = session.run(None, {input_name: batch_input}) uses correct shape and numpy float32 array. batch_input = torch.rand(5, 3, 224, 224) output = session.run(None, {input_name: batch_input}) uses torch tensor (not accepted). batch_input = np.random.rand(3, 224, 224).astype(np.float32) output = session.run(None, {input_name: batch_input}) misses batch dimension. batch_input = np.random.rand(5, 224, 224, 3).astype(np.float32) output = session.run(None, {input_name: batch_input}) swaps channel and width/height.

3. Final Answer:

   import numpy as np batch_input = np.random.rand(5, 3, 224, 224).astype(np.float32) output = session.run(None, {input_name: batch_input}) -> Option A

4. Quick Check:

   Correct shape and numpy array input [OK]

Hint: Match model input shape and use numpy float32 arrays [OK]

Common Mistakes:

• Using torch tensors instead of numpy arrays
• Incorrect input shape order
• Missing batch dimension