ORB features in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - ORB features

Which metric matters for ORB features and WHY

ORB features are used to find and match key points in images. The main metrics to check are matching accuracy and repeatability. Matching accuracy tells us how many correct matches the ORB detector finds between two images. Repeatability shows if ORB finds the same points when the image changes a bit (like rotation or lighting). These metrics matter because ORB is used in tasks like object recognition or tracking, where correct and stable matches are important.

Confusion matrix or equivalent visualization

For ORB feature matching, we can think of matches as either correct or incorrect. Here is a simple confusion matrix for matches:

      | Predicted Match | Predicted No Match |
      |-----------------|--------------------|
      | True Match (TP) | False Positive (FP) |
      | False Negative (FN)| True Negative (TN)  |
    

For example, if ORB finds 80 correct matches (TP), misses 20 true matches (FN), and finds 10 wrong matches (FP), we can calculate precision and recall to understand quality.

Precision vs Recall tradeoff with examples

Precision means how many of the matches ORB found are actually correct. High precision means fewer wrong matches.

Recall means how many of the true matches ORB was able to find. High recall means ORB finds most of the real matches.

Example: In a robot navigation task, high recall is important so the robot sees enough landmarks to localize itself. But too many wrong matches (low precision) can confuse it. So a balance is needed.

Adjusting ORB parameters (like number of features or matching threshold) changes this tradeoff.

What "good" vs "bad" metric values look like for ORB features

Good values:

• Precision above 0.8 means most matches are correct.
• Recall above 0.7 means most true matches are found.
• Repeatability above 0.75 means ORB finds stable points under changes.

Bad values:

• Precision below 0.5 means many wrong matches, which can cause errors.
• Recall below 0.4 means ORB misses many true points, reducing usefulness.
• Low repeatability means ORB points change a lot with small image changes.

Common pitfalls in ORB feature metrics

• Ignoring false matches: High number of wrong matches can look like good performance if only total matches are counted.
• Overfitting to one image pair: ORB parameters tuned for one pair may not work well on others.
• Data leakage: Using the same images for tuning and testing can give overly optimistic results.
• Ignoring image conditions: ORB performance drops with blur or lighting changes; metrics should test under varied conditions.

Self-check question

Your ORB feature matcher has 90% precision but only 30% recall on a set of image pairs. Is this good for a robot that needs to recognize places reliably? Why or why not?

Answer: This is not good because the low recall (30%) means ORB misses many true matches. The robot may fail to recognize places since it does not find enough correct points, even though the matches it finds are mostly correct (high precision). For reliable recognition, recall should be higher.

Key Result

For ORB features, balancing high precision and recall ensures correct and stable keypoint matches for reliable image tasks.