Re-ranking retrieved results in Prompt Engineering / GenAI - Model Pipeline Trace

Model Pipeline - Re-ranking retrieved results

This pipeline improves search results by first retrieving many items, then re-ordering them to show the best matches on top.

Data Flow - 5 Stages

1Initial Retrieval

1 query x large document database→Retrieve top 100 documents using basic search→100 documents

Query: 'best Italian restaurants' → Retrieved 100 restaurant descriptions

↓

2Feature Extraction

100 documents→Convert documents and query into vector features→100 vectors x 512 features

Each restaurant description converted to a 512-length vector

↓

3Re-ranking Model

100 vectors x 512 features→Score each document's relevance to query using a neural network→100 scores

Scores like 0.85, 0.78, 0.95 indicating relevance

↓

4Sort by Score

100 documents + 100 scores→Sort documents descending by score→100 documents ordered

Top document has score 0.95, second 0.90, etc.

↓

5Final Output

100 ordered documents→Return top 10 documents to user→10 documents

Top 10 restaurant descriptions shown to user

Training Trace - Epoch by Epoch

Loss
0.7 | *
0.6 | **
0.5 | ***
0.4 | ****
0.3 | *****
0.2 | ******
     ----------------
      1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.60	Model starts learning to rank documents better
2	0.48	0.72	Loss decreases, accuracy improves as model learns
3	0.35	0.81	Model shows good ranking ability
4	0.28	0.86	Further improvement, loss steadily decreases
5	0.22	0.90	Model converges with high accuracy

Prediction Trace - 4 Layers

Layer 1: Input Query and Retrieved Documents

Layer 2: Neural Network Scoring

Layer 3: Sorting

Layer 4: Select Top Results

Model Quiz - 3 Questions

Why do we convert documents into vectors before re-ranking?

A Because vectors remove all meaning from documents

B Because vectors make documents shorter

C Because vectors let the model compare documents numerically

D Because vectors are easier to read by humans

Key Insight

Re-ranking improves search by using a model to score and reorder results, making the best matches appear first. This step refines initial broad retrieval to a focused, relevant list.