First image processing program in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - First image processing program

Which metric matters for this concept and WHY

For a first image processing program, common tasks include detecting edges, colors, or simple shapes. The key metric to check is accuracy of the output compared to expected results. For example, if the program detects edges, accuracy means how many edges it found correctly versus missed or falsely detected. This helps us know if the program works as intended.

Confusion matrix or equivalent visualization (ASCII)

Imagine the program detects edges in an image. We can compare its output to a correct edge map and count:

      |               | Detected Edge | No Edge |
      |---------------|---------------|---------|
      | True Edge     | TP = 80       | FN = 15 |
      | True No Edge  | FP = 10       | TN = 95 |
    

Here, TP means edges correctly found, FP means wrong edges found, FN means edges missed, and TN means correctly ignored non-edges.

Precision vs Recall tradeoff with concrete examples

In edge detection:

• Precision = TP / (TP + FP): How many detected edges are actually real edges? High precision means few false edges.
• Recall = TP / (TP + FN): How many real edges did the program find? High recall means few missed edges.

If the program is too sensitive, it finds many edges but also many false ones (high recall, low precision). If it is too strict, it finds only very clear edges but misses some (high precision, low recall). Balancing these depends on what matters more: not missing edges or not adding false edges.

What "good" vs "bad" metric values look like for this use case

Good edge detection program metrics might be:

• Precision around 0.85 or higher (most detected edges are real)
• Recall around 0.80 or higher (most real edges are found)
• F1 score (balance of precision and recall) above 0.80

Bad metrics would be:

• Precision below 0.5 (many false edges)
• Recall below 0.5 (many missed edges)
• F1 score below 0.5 (poor overall detection)

Metrics pitfalls

• Accuracy paradox: If most pixels are non-edges, a program that detects no edges can have high accuracy but be useless.
• Data leakage: Testing on images the program already saw can give falsely high metrics.
• Overfitting: Program tuned too much on one image type may fail on others, showing good metrics only on training images.

Self-check question

Your edge detection program has 98% accuracy but only 12% recall on edges. Is it good?

Answer: No. The high accuracy is misleading because most pixels are non-edges. The very low recall means it misses almost all real edges, so it does not work well.

Key Result

For first image processing programs, balance precision and recall to ensure meaningful detection beyond simple accuracy.