Homography and image alignment in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Homography and image alignment

Which metric matters for Homography and image alignment and WHY

For homography and image alignment, the key metric is the Reprojection Error. This measures how far the points from one image move when transformed to the other image using the estimated homography. A smaller reprojection error means the alignment is more accurate. This metric directly shows how well the model matches points between images, which is the goal of alignment.

Other useful metrics include Inlier Ratio from RANSAC, which tells us how many matched points fit the homography well, indicating robustness.

Confusion matrix or equivalent visualization

Homography estimation does not use a confusion matrix like classification. Instead, we visualize alignment quality with a point matching diagram or reprojection error histogram.

    Example reprojection error histogram:
    -------------------------------------
    | Error Range | Number of Points     |
    | 0-1 pixel   | 150 (inliers)        |
    | 1-3 pixels  | 30                   |
    | >3 pixels | 20 (outliers)         |
    -------------------------------------
    

This shows how many points align closely (inliers) versus poorly (outliers).

Precision vs Recall tradeoff with concrete examples

In homography, the tradeoff is between inlier precision and inlier recall during point matching:

• High precision: Most matched points are correct (few false matches). This leads to a more accurate homography but might miss some true matches.
• High recall: Most true matches are found, but some false matches may be included, risking a less accurate homography.

For example, if you set a strict threshold in RANSAC, you get high precision but lower recall. If you loosen it, recall improves but precision drops.

Choosing the right balance depends on the application. For stitching photos, high precision avoids visible misalignments. For augmented reality, high recall ensures enough points for stable tracking.

What "good" vs "bad" metric values look like for this use case

Good alignment:

• Reprojection error < 1 pixel on average
• Inlier ratio > 80%
• Visual check shows overlapping images aligned well without ghosting

Bad alignment:

• Reprojection error > 5 pixels on average
• Inlier ratio < 50%
• Images show visible misalignment or double edges

Metrics pitfalls

• Ignoring outliers: Including many wrong matches can lower homography quality but may not be obvious if only average error is reported.
• Overfitting to noise: A homography that fits noisy points too closely may have low error but poor generalization.
• Data leakage: Using the same points to estimate and evaluate homography inflates performance metrics.
• Ignoring scale and rotation: Metrics should consider geometric transformations, not just pixel distance.

Self-check question

Your homography model shows an average reprojection error of 0.8 pixels but only 40% inlier ratio. Is this good?

Answer: Not really. While the low reprojection error means the matched points fit well, the low inlier ratio means most points do not fit the homography. This suggests the model is only good for a small subset of points and may not align the images well overall. You should improve matching or outlier rejection.

Key Result

Reprojection error and inlier ratio are key metrics; low error with high inlier ratio indicates good homography alignment.