ONNX Runtime in Computer Vision - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

ONNX Runtime Model Inference Output

What is the output shape of the prediction when running this ONNX Runtime code snippet on a model expecting input shape (1, 3, 224, 224)?

Computer Vision

import onnxruntime as ort
import numpy as np

session = ort.InferenceSession('model.onnx')
input_name = session.get_inputs()[0].name
input_data = np.random.randn(1, 3, 224, 224).astype(np.float32)
outputs = session.run(None, {input_name: input_data})
prediction = outputs[0]
print(prediction.shape)

A (1, 1000)

B (224, 224, 3)

C (3, 224, 224)

D (1000, 1)

💡 Hint

The model outputs class probabilities for 1000 classes in a batch of 1.

The model expects a batch of images with shape (1, 3, 224, 224). The output is typically a batch of predictions with shape (batch_size, number_of_classes), here (1, 1000).

❓ Model Choice

intermediate

Choosing ONNX Runtime Execution Provider

Which ONNX Runtime execution provider should you choose to maximize inference speed on a machine with an NVIDIA GPU?

A CPUExecutionProvider

B CUDAExecutionProvider

C TensorRTExecutionProvider

D OpenVINOExecutionProvider

💡 Hint

NVIDIA GPUs are best supported by a specific provider.

CUDAExecutionProvider uses NVIDIA GPU acceleration for faster inference. CPUExecutionProvider runs on CPU only. OpenVINO is for Intel hardware. TensorRT is also for NVIDIA but requires extra setup.

❓ Hyperparameter

advanced

Batch Size Effect on ONNX Runtime Performance

Increasing the batch size during ONNX Runtime inference usually has which effect on throughput and latency?

A Throughput increases, latency increases

B Throughput decreases, latency decreases

C Throughput decreases, latency increases

D Throughput increases, latency decreases

💡 Hint

Think about processing more inputs at once versus time per input.

Larger batch sizes allow more inputs processed per second (higher throughput) but each batch takes longer to process (higher latency).

🔧 Debug

advanced

ONNX Runtime Input Type Error

What error will this ONNX Runtime code raise when passing input data as a Python list instead of a numpy array?

Computer Vision

import onnxruntime as ort

session = ort.InferenceSession('model.onnx')
input_name = session.get_inputs()[0].name
input_data = [[0.5]*224]*224  # Python list, not numpy array
outputs = session.run(None, {input_name: input_data})

A ValueError: Input shape mismatch

B No error, runs successfully

C RuntimeError: Model file not found

D TypeError: Expected numpy.ndarray for input

💡 Hint

ONNX Runtime expects inputs as numpy arrays with correct dtype.

Passing a Python list instead of a numpy array causes a TypeError because ONNX Runtime requires numpy.ndarray inputs.

🧠 Conceptual

expert

ONNX Runtime Model Optimization

Which ONNX Runtime feature allows you to optimize a model by fusing nodes and eliminating redundant operations to improve inference speed?

A Execution Provider Selection

B Dynamic Batching

C Graph Optimization Level

D Model Quantization

💡 Hint

This feature works on the model graph before running inference.

Graph Optimization Level controls how ONNX Runtime optimizes the model graph by fusing nodes and removing redundancies to speed up inference.

Practice

1. What is the main purpose of ONNX Runtime in machine learning?

easy

A. To collect and label training data

B. To train new machine learning models from scratch

C. To visualize data and create charts

D. To run pre-trained machine learning models efficiently on different devices

Solution

1. Step 1: Understand ONNX Runtime's role

   ONNX Runtime is designed to run models that are already trained, not to train new ones.

2. Step 2: Identify the correct purpose

   It helps run these models efficiently on many devices, making deployment easier.

3. Final Answer:

   To run pre-trained machine learning models efficiently on different devices -> Option D

4. Quick Check:

   ONNX Runtime runs models = A [OK]

Hint: ONNX Runtime runs models, not trains them [OK]

Common Mistakes:

• Confusing ONNX Runtime with training frameworks
• Thinking it is for data visualization
• Assuming it collects or labels data

2. Which Python code snippet correctly loads an ONNX model using ONNX Runtime?

easy

A. import onnxruntime as ort
session = ort.Model('model.onnx')

B. import onnxruntime as ort
session = ort.load_model('model.onnx')

C. import onnxruntime as ort
session = ort.InferenceSession('model.onnx')

D. import onnxruntime as ort
session = ort.run('model.onnx')

Solution

1. Step 1: Recall ONNX Runtime loading method

   The correct method to load a model is using InferenceSession with the model file path.

2. Step 2: Check each option

   Only import onnxruntime as ort session = ort.InferenceSession('model.onnx') uses ort.InferenceSession correctly; others use invalid methods.

3. Final Answer:

   import onnxruntime as ort\nsession = ort.InferenceSession('model.onnx') -> Option C

4. Quick Check:

   Use InferenceSession to load model = A [OK]

Hint: Use ort.InferenceSession('model.onnx') to load model [OK]

Common Mistakes:

• Using non-existent methods like load_model or run
• Not importing onnxruntime correctly
• Confusing model loading with running

3. Given the code below, what will be the output type of outputs?

import onnxruntime as ort
import numpy as np

session = ort.InferenceSession('model.onnx')
input_name = session.get_inputs()[0].name
input_data = np.random.rand(1, 3, 224, 224).astype(np.float32)
outputs = session.run(None, {input_name: input_data})
print(type(outputs))

medium

A.

B.

C.

D.

Solution

1. Step 1: Understand session.run output

   Calling session.run returns a list of outputs from the model.

2. Step 2: Check the print statement

   Printing type(outputs) will show <class 'list'> because outputs is a list.

3. Final Answer:

   <class 'list'> -> Option A

4. Quick Check:

   session.run returns list = C [OK]

Hint: session.run returns a list of outputs [OK]

Common Mistakes:

• Assuming outputs is a numpy array directly
• Thinking outputs is a dictionary
• Confusing tuple with list

4. Identify the error in the following ONNX Runtime code snippet:

import onnxruntime as ort
session = ort.InferenceSession('model.onnx')
input_name = session.get_inputs()[0]
input_data = [1.0, 2.0, 3.0]
outputs = session.run(None, {input_name: input_data})

medium

A. input_name should be the name string, not the input object

B. input_data must be a dictionary, not a list

C. session.run requires the model path as first argument

D. onnxruntime does not support list inputs

Solution

1. Step 1: Check input_name assignment

   session.get_inputs()[0] returns an input object, but session.run expects the input name string as key.

2. Step 2: Correct usage

   Use session.get_inputs()[0].name to get the input name string for the dictionary key.

3. Final Answer:

   input_name should be the name string, not the input object -> Option A

4. Quick Check:

   Use input_name = session.get_inputs()[0].name [OK]

Hint: Use input_name = session.get_inputs()[0].name [OK]

Common Mistakes:

• Using input object instead of input name string
• Passing wrong input data types
• Misunderstanding session.run arguments

5. You want to run an ONNX model on a GPU using ONNX Runtime. Which code snippet correctly enables GPU execution?

hard

A. import onnxruntime as ort
session = ort.InferenceSession('model.onnx', execution_mode='GPU')

B. import onnxruntime as ort
session = ort.InferenceSession('model.onnx', providers=['CUDAExecutionProvider'])

C. import onnxruntime as ort
session = ort.InferenceSession('model.onnx', use_gpu=True)

D. import onnxruntime as ort
session = ort.InferenceSession('model.onnx', device='GPU')

Solution

1. Step 1: Recall how to enable GPU in ONNX Runtime

   ONNX Runtime uses the 'providers' argument with 'CUDAExecutionProvider' to run on GPU.

2. Step 2: Check each option

   Only import onnxruntime as ort session = ort.InferenceSession('model.onnx', providers=['CUDAExecutionProvider']) correctly uses providers=['CUDAExecutionProvider']; others use invalid parameters.

3. Final Answer:

   import onnxruntime as ort\nsession = ort.InferenceSession('model.onnx', providers=['CUDAExecutionProvider']) -> Option B

4. Quick Check:

   Use providers=['CUDAExecutionProvider'] for GPU [OK]

Hint: Set providers=['CUDAExecutionProvider'] to use GPU [OK]

Common Mistakes:

• Using non-existent parameters like device or use_gpu
• Confusing execution_mode with providers
• Not specifying providers disables GPU