Open-domain QA basics in NLP - Model Pipeline Trace

Model Pipeline - Open-domain QA basics

This pipeline answers questions by searching a large collection of documents and then selecting the best answer. It first finds relevant text pieces, then reads them carefully to find the exact answer.

Data Flow - 5 Stages

1Input Question

1 question string→Receive a natural language question from user→1 question string

"What is the capital of France?"

↓

2Document Retrieval

1 question string→Search a large text database to find top relevant documents or passages→5 passages x 100 words each

["Paris is the capital city of France...", "France's largest city is Paris..."]

↓

3Context Preparation

5 passages x 100 words each→Combine retrieved passages into a single context for reading→1 context string (~500 words)

"Paris is the capital city of France. It is known for... France's largest city is Paris..."

↓

4Answer Extraction Model

1 question string + 1 context string→Use a reading comprehension model to find answer span in context→1 answer string

"Paris"

↓

5Output Answer

1 answer string→Return the extracted answer to the user→1 answer string

"Paris"

Training Trace - Epoch by Epoch

Loss
1.2 |****
1.0 |*** 
0.8 |**  
0.6 |*   
0.4 |    
     +----
      1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Model starts learning to locate answers in text.
2	0.9	0.60	Model improves understanding of question and context.
3	0.7	0.72	Model better identifies correct answer spans.
4	0.5	0.80	Model converges with good answer extraction ability.
5	0.4	0.85	Final fine-tuning improves accuracy slightly.

Prediction Trace - 5 Layers

Layer 1: Input Question

Layer 2: Document Retrieval

Layer 3: Context Preparation

Layer 4: Answer Extraction Model

Layer 5: Output Answer

Model Quiz - 3 Questions

What is the main role of the Document Retrieval stage?

A Generate the final answer directly

B Find relevant text passages related to the question

C Combine passages into one context

D Train the model to understand questions

Key Insight

Open-domain QA works by first finding relevant information, then carefully reading it to pick the exact answer. Training improves the model's ability to locate answers, shown by decreasing loss and increasing accuracy.