Feature matching between images in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Feature matching between images

Which metric matters for feature matching and WHY

In feature matching between images, the key metrics are matching accuracy and matching precision. These tell us how many matched points are correct versus incorrect. Since feature matching finds pairs of points that represent the same real-world spot, we want to measure how many matches are true (correct) and how many are false (wrong). High precision means most matches are correct, which is important to avoid wrong matches that confuse later steps like 3D reconstruction or stitching.

Confusion matrix for feature matching

      | Predicted Match    | Predicted No Match |
      |--------------------|--------------------|
      | True Positive (TP)  | False Negative (FN) |
      | False Positive (FP) | True Negative (TN)  |

      TP: Correctly matched points
      FP: Incorrectly matched points
      FN: Missed correct matches
      TN: Correctly rejected non-matches
    

For example, if we have 100 true matching points, and our algorithm finds 90 matches, 80 of which are correct (TP=80), 10 wrong (FP=10), and misses 20 (FN=20), then:

• Precision = 80 / (80 + 10) = 0.89
• Recall = 80 / (80 + 20) = 0.80

Precision vs Recall tradeoff with examples

Feature matching often balances precision and recall:

• High precision, low recall: Matches are mostly correct but many true matches are missed. Useful when wrong matches cause big problems, like in 3D modeling where errors ruin the model.
• High recall, low precision: Most true matches are found but many wrong matches appear. This might be okay for rough alignment but can cause errors downstream.

For example, in panorama stitching, high precision avoids visible ghosting from wrong matches. In object recognition, high recall ensures the object is detected even if some matches are noisy.

What good vs bad metric values look like

Good feature matching results:

• Precision above 0.85 means most matches are correct.
• Recall above 0.75 means most true matches are found.
• Balanced precision and recall around 0.8 or higher is ideal.

Bad results:

• Precision below 0.5 means many wrong matches, causing errors.
• Recall below 0.4 means many true matches are missed, losing information.
• Very high recall but very low precision means noisy matches.

Common pitfalls in feature matching metrics

• Ignoring false matches: Only counting total matches without checking correctness can be misleading.
• Data leakage: Using the same images for tuning and testing inflates metrics.
• Overfitting: Matching tuned to specific image pairs may fail on new images.
• Accuracy paradox: High overall match count but low precision means many wrong matches.

Self-check question

Your feature matching model finds 95% of true matches (recall = 0.95) but only 40% of its matches are correct (precision = 0.40). Is this good for production? Why or why not?

Answer: No, this is not good. Although the model finds most true matches, it also produces many wrong matches (low precision). Wrong matches can cause errors in later steps like stitching or 3D reconstruction. You want to improve precision to reduce false matches.

Key Result

Precision and recall are key to evaluate feature matching; high precision avoids wrong matches, high recall finds most true matches.