Vector similarity metrics in Prompt Engineering / GenAI - Model Metrics & Evaluation

Metrics & Evaluation - Vector similarity metrics

Which metric matters for Vector similarity metrics and WHY

Vector similarity metrics measure how alike two vectors are. They help find items that are close or related in meaning or features. Common metrics include Cosine similarity, Euclidean distance, and Manhattan distance.

Cosine similarity is popular because it measures the angle between vectors, ignoring their length. This is useful when direction matters more than size, like comparing text meanings.

Euclidean distance measures straight-line distance between points, useful when absolute difference matters.

Choosing the right metric depends on your data and what "similar" means in your task.

Confusion matrix or equivalent visualization

Vector similarity does not use a confusion matrix like classification. Instead, we look at similarity scores between pairs.

    Example: Comparing query vector Q with database vectors A, B, C

    Vector pairs and Cosine similarity scores:
    Q & A: 0.95 (very similar)
    Q & B: 0.60 (somewhat similar)
    Q & C: 0.10 (not similar)

    Higher scores mean more similarity (max 1.0).
    

Precision vs Recall tradeoff with concrete examples

When using vector similarity for search or recommendations, you pick a similarity threshold to decide what counts as "similar enough."

High threshold (e.g., 0.9): Only very close matches are returned. This means high precision (few wrong matches) but low recall (may miss some relevant items).

Low threshold (e.g., 0.5): More items are returned, including less similar ones. This means high recall (finds most relevant items) but lower precision (more irrelevant items included).

Example: In a movie recommendation system, a high threshold shows only very similar movies (precise but fewer), while a low threshold shows many movies including less related ones.

What "good" vs "bad" metric values look like for Vector similarity

Good: Similar items have high similarity scores (close to 1 for cosine), and dissimilar items have low scores (close to 0 or negative for cosine). Clear separation helps make confident decisions.

Bad: Scores cluster around the middle (e.g., 0.5) for all pairs, making it hard to tell similar from dissimilar. This means the metric or vector representation is not capturing meaningful differences.

Metrics pitfalls

• Ignoring vector normalization: Cosine similarity does not require vectors to be normalized; it inherently measures the angle between vectors regardless of their length. However, normalizing vectors can improve numerical stability.
• Using Euclidean distance on high-dimensional sparse data: Can cause "curse of dimensionality" where distances become less meaningful.
• Choosing wrong metric for data type: For example, cosine similarity is better for text embeddings, but Euclidean might be better for physical coordinates.
• Threshold selection without validation: Picking similarity cutoffs without testing can lead to poor precision or recall.

Self-check question

Your search system uses cosine similarity with a threshold of 0.8. You find many relevant results but also many irrelevant ones. What should you do?

Answer: Lowering the threshold will increase recall but reduce precision, so to reduce irrelevant results, you should raise the threshold above 0.8 to get fewer but more precise matches.

Key Result

Vector similarity metrics like cosine similarity measure how close vectors are; choosing the right metric and threshold balances precision and recall in similarity tasks.