RAG architecture overview in Prompt Engineering / GenAI - Model Pipeline Trace

Model Pipeline - RAG architecture overview

The RAG (Retrieval-Augmented Generation) architecture combines a search step with a text generation step. It first finds relevant documents from a large collection, then uses those documents to help create better answers.

Data Flow - 4 Stages

1Input Query

1 query string→User provides a question or prompt→1 query string

"What is the capital of France?"

↓

2Document Retrieval

1 query string→Searches a large document database for relevant texts→Top 5 documents (5 texts)

["Paris is the capital of France.", "France is in Europe.", "The Eiffel Tower is in Paris.", "French culture is rich.", "Paris has many museums."]

↓

3Context Preparation

1 query string + 5 documents→Combines query with retrieved documents to form input for generation→1 combined input string

"Question: What is the capital of France? Context: Paris is the capital of France. France is in Europe. The Eiffel Tower is in Paris. French culture is rich. Paris has many museums."

↓

4Text Generation

1 combined input string→Generates an answer using a language model conditioned on the context→1 answer string

"The capital of France is Paris."

Training Trace - Epoch by Epoch

Loss: 2.3 |****     
       1.8 |******   
       1.4 |******** 
       1.1 |*********
       0.9 |*********

Epoch	Loss ↓	Accuracy ↑	Observation
1	2.3	0.25	Model starts learning, loss is high, accuracy low
2	1.8	0.40	Loss decreases, accuracy improves
3	1.4	0.55	Model learns better context usage
4	1.1	0.65	Continued improvement in generation quality
5	0.9	0.72	Model converges with good retrieval and generation

Prediction Trace - 4 Layers

Layer 1: Input Query

Layer 2: Document Retrieval

Layer 3: Context Preparation

Layer 4: Text Generation

Model Quiz - 3 Questions

What is the main role of the document retrieval step in RAG?

A Find relevant documents to help answer the query

B Generate the final answer text

C Clean the input query

D Evaluate model accuracy

Key Insight

RAG improves answer quality by first finding helpful documents, then using them as context for generation. This two-step approach helps the model give more accurate and informative responses.