ONNX Runtime in Computer Vision - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

ONNX Runtime Model Inference Output

What is the output shape of the prediction when running this ONNX Runtime code snippet on a model expecting input shape (1, 3, 224, 224)?

Computer Vision

import onnxruntime as ort
import numpy as np

session = ort.InferenceSession('model.onnx')
input_name = session.get_inputs()[0].name
input_data = np.random.randn(1, 3, 224, 224).astype(np.float32)
outputs = session.run(None, {input_name: input_data})
prediction = outputs[0]
print(prediction.shape)

A (1, 1000)

B (224, 224, 3)

C (3, 224, 224)

D (1000, 1)

💡 Hint

The model outputs class probabilities for 1000 classes in a batch of 1.

The model expects a batch of images with shape (1, 3, 224, 224). The output is typically a batch of predictions with shape (batch_size, number_of_classes), here (1, 1000).

❓ Model Choice

intermediate

Choosing ONNX Runtime Execution Provider

Which ONNX Runtime execution provider should you choose to maximize inference speed on a machine with an NVIDIA GPU?

A CPUExecutionProvider

B CUDAExecutionProvider

C TensorRTExecutionProvider

D OpenVINOExecutionProvider

💡 Hint

NVIDIA GPUs are best supported by a specific provider.

CUDAExecutionProvider uses NVIDIA GPU acceleration for faster inference. CPUExecutionProvider runs on CPU only. OpenVINO is for Intel hardware. TensorRT is also for NVIDIA but requires extra setup.

❓ Hyperparameter

advanced

Batch Size Effect on ONNX Runtime Performance

Increasing the batch size during ONNX Runtime inference usually has which effect on throughput and latency?

A Throughput increases, latency increases

B Throughput decreases, latency decreases

C Throughput decreases, latency increases

D Throughput increases, latency decreases

💡 Hint

Think about processing more inputs at once versus time per input.

Larger batch sizes allow more inputs processed per second (higher throughput) but each batch takes longer to process (higher latency).

🔧 Debug

advanced

ONNX Runtime Input Type Error

What error will this ONNX Runtime code raise when passing input data as a Python list instead of a numpy array?

Computer Vision

import onnxruntime as ort

session = ort.InferenceSession('model.onnx')
input_name = session.get_inputs()[0].name
input_data = [[0.5]*224]*224  # Python list, not numpy array
outputs = session.run(None, {input_name: input_data})

A ValueError: Input shape mismatch

B No error, runs successfully

C RuntimeError: Model file not found

D TypeError: Expected numpy.ndarray for input

💡 Hint

ONNX Runtime expects inputs as numpy arrays with correct dtype.

Passing a Python list instead of a numpy array causes a TypeError because ONNX Runtime requires numpy.ndarray inputs.

🧠 Conceptual

expert

ONNX Runtime Model Optimization

Which ONNX Runtime feature allows you to optimize a model by fusing nodes and eliminating redundant operations to improve inference speed?

A Execution Provider Selection

B Dynamic Batching

C Graph Optimization Level

D Model Quantization

💡 Hint

This feature works on the model graph before running inference.

Graph Optimization Level controls how ONNX Runtime optimizes the model graph by fusing nodes and removing redundancies to speed up inference.