Question answering in Prompt Engineering / GenAI - Model Pipeline Trace

Model Pipeline - Question answering

This pipeline takes a question and a related text passage as input. It processes the text to understand the context, then uses a trained model to find the answer to the question within the passage.

Data Flow - 6 Stages

1Input data

1000 samples x 2 texts (question and passage)→Receive pairs of questions and passages→1000 samples x 2 texts

Question: 'What color is the sky?' Passage: 'The sky is blue on a clear day.'

↓

2Preprocessing

1000 samples x 2 texts→Tokenize and convert texts to numerical tokens→1000 samples x 2 sequences of tokens (max length 128)

Question tokens: [101, 2054, 2079, 2003, 1996, 3710, 102] Passage tokens: [101, 1996, 3710, 2003, 2631, 2006, 1037, 2706, 2154, 1012, 102]

↓

3Feature Engineering

1000 samples x 2 sequences of tokens→Create input embeddings combining question and passage tokens→1000 samples x 128 tokens x 768 features

Embedding vector for token 101: [0.12, -0.05, ..., 0.33]

↓

4Model Training

1000 samples x 128 tokens x 768 features→Train transformer-based QA model to predict answer span→1000 samples x 2 indices (start and end positions)

Predicted answer span: start=4, end=4 (tokens corresponding to 'blue')

↓

5Evaluation

1000 samples x 2 indices→Calculate exact match and F1 score comparing predicted and true answers→Metrics: exact match=0.78, F1=0.85

Exact match: 78% of predicted answers exactly match true answers

↓

6Prediction

1 sample x 2 texts→Model predicts answer span for new question and passage→1 sample x answer text

Question: 'What color is the sky?' Answer: 'blue'

Training Trace - Epoch by Epoch

Loss
1.2 |****
0.9 |***
0.7 |**
0.5 |*
0.4 |

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Model starts learning, loss high, accuracy low
2	0.9	0.60	Loss decreases, accuracy improves
3	0.7	0.70	Model learns better answer spans
4	0.5	0.78	Loss continues to decrease, accuracy rises
5	0.4	0.82	Model converges with good performance

Prediction Trace - 5 Layers

Layer 1: Tokenization

Layer 2: Embedding Layer

Layer 3: Transformer Encoder

Layer 4: Answer Span Prediction

Layer 5: Answer Extraction

Model Quiz - 3 Questions

What does the model predict as output after training?

A A yes or no answer

B A summary of the passage

C Start and end positions of the answer in the passage

D The question rewritten

Key Insight

This visualization shows how a question answering model learns to find exact answer spans in text passages by converting words into meaningful vectors and improving predictions over training epochs.

Practice

1. What is the main purpose of question answering in AI?

easy

A. To find answers from given text or context

B. To generate random text without context

C. To translate languages automatically

D. To create images from descriptions

Solution

1. Step 1: Understand the goal of question answering

   Question answering systems are designed to find specific answers from a given text or context.

2. Step 2: Compare options with the goal

   Only To find answers from given text or context describes finding answers from text, which matches the purpose.

3. Final Answer:

   To find answers from given text or context -> Option A

4. Quick Check:

   Question answering = find answers [OK]

Hint: Focus on 'answer from text' meaning [OK]

Common Mistakes:

• Confusing question answering with translation
• Thinking it generates random text
• Mixing it with image generation

2. Which input is essential for a question answering model to work?

easy

A. Only a context without a question

B. Only a question without any context

C. A question and a related context or passage

D. Random text unrelated to the question

Solution

1. Step 1: Identify inputs needed for question answering

   Question answering requires both a question and some context to find the answer.

2. Step 2: Match options with required inputs

   Only A question and a related context or passage provides both question and related context, which is necessary.

3. Final Answer:

   A question and a related context or passage -> Option C

4. Quick Check:

   Question + context = answer [OK]

Hint: Remember: question needs context to answer [OK]

Common Mistakes:

• Assuming question alone is enough
• Ignoring the need for context
• Choosing unrelated text as input

3. Given this Python code using a question answering model:

from transformers import pipeline
qa = pipeline('question-answering')
context = "The Eiffel Tower is in Paris."
question = "Where is the Eiffel Tower located?"
result = qa(question=question, context=context)
print(result['answer'])

What will be printed?

medium

A. Location unknown

B. Eiffel Tower

C. The Eiffel Tower is in Paris

D. Paris

Solution

1. Step 1: Understand the code's purpose

   The code uses a question answering pipeline to find the answer to the question from the context.

2. Step 2: Identify the answer in the context

   The question asks for location; the context says "The Eiffel Tower is in Paris." So the answer is "Paris".

3. Final Answer:

   Paris -> Option D

4. Quick Check:

   Answer extracted = Paris [OK]

Hint: Look for direct answer in context matching question [OK]

Common Mistakes:

• Printing the whole context instead of answer
• Confusing object with location
• Assuming no answer found

4. This code snippet tries to answer a question but raises an error:

from transformers import pipeline
qa = pipeline('question-answering')
context = "Python is a programming language."
question = "What is Python?"
result = qa(question, context)
print(result['answer'])

What is the error and how to fix it?

medium

A. Error: question is invalid; fix by changing question text

B. Error: missing keyword arguments; fix by using qa(question=question, context=context)

C. Error: context is empty; fix by adding text to context

D. No error; code runs fine

Solution

1. Step 1: Identify the function call error

   The pipeline expects keyword arguments question= and context=, but code passes positional arguments.

2. Step 2: Fix the call with correct keywords

   Change to qa(question=question, context=context) to fix the error.

3. Final Answer:

   Error: missing keyword arguments; fix by using qa(question=question, context=context) -> Option B

4. Quick Check:

   Use keywords for qa() args [OK]

Hint: Use keyword arguments for question and context [OK]

Common Mistakes:

• Passing positional args instead of keywords
• Assuming empty context causes error
• Changing question text unnecessarily

5. You want to build a question answering system that can handle multiple paragraphs and find the best answer. Which approach is best?

hard

A. Split text into paragraphs, run QA on each, then pick highest confidence answer

B. Combine all paragraphs into one string and run QA once

C. Only use the first paragraph for QA

D. Ignore paragraphs and guess answer randomly

Solution

1. Step 1: Understand handling multiple paragraphs

   QA models usually work best on smaller text chunks, so splitting helps.

2. Step 2: Choose method to find best answer

   Running QA on each paragraph separately and selecting the answer with highest confidence ensures accuracy.

3. Final Answer:

   Split text into paragraphs, run QA on each, then pick highest confidence answer -> Option A

4. Quick Check:

   Split + score answers = best result [OK]

Hint: Split text, run QA per part, pick best answer [OK]

Common Mistakes:

• Running QA on all text at once causing confusion
• Ignoring paragraphs reduces accuracy
• Guessing answers without context