Extractive QA concept in NLP - Deep Dive

Overview - Extractive QA concept

What is it?

Extractive Question Answering (QA) is a method where a system finds the exact answer to a question by selecting a piece of text from a given document or passage. Instead of generating new text, it extracts the answer directly from the source. This helps machines understand and respond to questions using existing information.

Why it matters

Extractive QA solves the problem of quickly finding precise answers from large amounts of text, like articles or reports. Without it, people would have to read everything themselves, which is slow and tiring. It powers search engines, virtual assistants, and customer support by giving fast, accurate answers.

Where it fits

Before learning Extractive QA, you should understand basic natural language processing concepts like tokenization and embeddings. After this, you can explore generative QA, where answers are created rather than extracted, and advanced models like transformers for better understanding.

Mental Model

Core Idea

Extractive QA works by scanning a text to find the exact part that answers a question, like highlighting a sentence in a book.

Think of it like...

Imagine you have a big book and someone asks you a question. Instead of rewriting the answer, you flip through the pages and point to the exact sentence that answers them.

┌───────────────┐
│   Question    │
└──────┬────────┘
       │
       ▼
┌─────────────────────────────┐
│       Text Passage           │
│  ┌───────────────────────┐  │
│  │  Extracted Answer     │  │
│  │  (Exact text snippet) │  │
│  └───────────────────────┘  │
└─────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Questions and Text

Concept: Learn what questions and text passages are in simple terms and how they relate.

A question is something you want to know, like 'What is the capital of France?'. A text passage is a piece of writing that might contain the answer, like a paragraph about France. Extractive QA finds the answer inside this passage.

Result

You can identify a question and a text passage that might contain the answer.

Understanding the basic elements of question and text is essential before trying to find answers inside text.

2

Foundation

What Does Extractive Mean?

Concept: Extractive means taking out something exactly as it is, without changing it.

In Extractive QA, the system does not create new sentences. It picks the exact words or sentences from the text that answer the question. This is different from making up an answer.

Result

You know that extractive means copying the answer directly from the text.

Knowing that answers come directly from the text helps set expectations for how the system works.

3

Intermediate

How Models Find Answers in Text

🤔Before reading on: do you think the model reads the whole text at once or word by word? Commit to your answer.

Concept: Models scan the entire text passage to find the start and end positions of the answer span.

Modern Extractive QA models look at the whole passage and the question together. They predict where the answer starts and ends by assigning scores to each word or token. The highest scoring span is chosen as the answer.

Result

The model outputs the exact snippet from the text that answers the question.

Understanding that models predict answer boundaries explains why they can find precise answers instead of guessing.

4

Intermediate

Role of Contextual Word Representations

🤔Before reading on: do you think the model treats each word the same or considers surrounding words? Commit to your answer.

Concept: Contextual word representations help the model understand words based on their surrounding words.

Words can have different meanings depending on context. For example, 'bank' can mean river edge or money place. Models use techniques like transformers to create word vectors that change meaning based on context, helping find the right answer.

Result

The model better understands the question and passage, improving answer accuracy.

Knowing that word meaning depends on context is key to why modern QA models work well.

5

Intermediate

Training Extractive QA Models

🤔Before reading on: do you think models learn from questions only or from questions with answers? Commit to your answer.

Concept: Models learn by seeing many examples of questions paired with exact answer spans in text.

During training, the model is given a question, a passage, and the correct answer span. It adjusts itself to predict the right start and end positions. Over time, it gets better at spotting answers in new texts.

Result

The model can generalize to find answers in unseen passages.

Understanding training with answer spans clarifies how models improve and why labeled data is important.

6

Advanced

Handling No-Answer Questions

🤔Before reading on: do you think every question always has an answer in the text? Commit to your answer.

Concept: Some questions may not have an answer in the given passage, and models must detect this.

Advanced Extractive QA models include a special option to say 'no answer' if the passage doesn't contain the answer. This prevents wrong guesses and improves reliability.

Result

The model can correctly say when no answer is found, avoiding misleading outputs.

Knowing how models handle no-answer cases is crucial for real-world applications where not all questions are answerable.

7

Expert

Limitations and Biases in Extractive QA

🤔Before reading on: do you think Extractive QA models always find the correct answer if it is in the text? Commit to your answer.

Concept: Extractive QA models can fail due to ambiguous questions, biased training data, or complex language.

Models sometimes pick wrong spans if the question is unclear or the text is tricky. They also reflect biases from training data, like favoring common answers. Understanding these limits helps improve and trust models.

Result

You recognize when and why models might make mistakes.

Awareness of model limitations guides better use and development of Extractive QA systems.

Under the Hood

Extractive QA models use deep neural networks, often transformers, to encode both the question and passage into vectors. They then compute scores for each token's likelihood of being the start or end of the answer span. The highest scoring span is selected as the answer. During training, the model learns to assign high scores to correct spans by minimizing a loss function comparing predictions to true answers.

Why designed this way?

Extractive QA was designed to provide precise answers without generating new text, reducing errors and hallucinations common in generative models. Using start and end positions simplifies the problem to span prediction, which is easier to train and interpret. Transformers were chosen for their ability to capture context and relationships in text effectively.

┌───────────────┐      ┌───────────────┐
│   Question    │      │  Text Passage  │
└──────┬────────┘      └──────┬────────┘
       │                      │
       │                      │
       ▼                      ▼
  ┌─────────────────────────────────┐
  │      Transformer Encoder         │
  │  (jointly encodes question &    │
  │   passage into contextual tokens)│
  └──────────────┬──────────────────┘
                 │
                 ▼
       ┌─────────────────────┐
       │ Start & End Scorers  │
       │ (predict answer span)│
       └─────────┬───────────┘
                 │
                 ▼
       ┌─────────────────────┐
       │ Extracted Answer Span│
       └─────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does Extractive QA generate new sentences as answers? Commit to yes or no.

Common Belief: Extractive QA creates new answers by writing sentences based on the question.

Reality: Extractive QA only selects exact text spans from the given passage; it does not generate new text.

Why it matters: Believing it generates text can lead to expecting creative or summarized answers, causing confusion when only exact text is returned.

Quick: Do you think Extractive QA always finds the correct answer if it exists in the text? Commit to yes or no.

Common Belief: If the answer is in the text, the model will always find it correctly.

Reality: Models can make mistakes due to ambiguous questions, complex language, or insufficient training data.

Why it matters: Overtrusting models can cause users to accept wrong answers, leading to misinformation or errors.

Quick: Does Extractive QA require the entire document to answer a question? Commit to yes or no.

Common Belief: Extractive QA models need the full document to find answers.

Reality: Models usually work on smaller passages or chunks because processing very long texts is inefficient and less accurate.

Why it matters: Expecting full-document processing can cause performance issues and misunderstandings about model capabilities.

Quick: Can Extractive QA handle questions with no answers in the text? Commit to yes or no.

Common Belief: Extractive QA always returns an answer, even if the text doesn't contain it.

Reality: Advanced models can detect no-answer cases and return 'no answer' instead of guessing.

Why it matters: Knowing this prevents accepting false answers when the text lacks relevant information.

Expert Zone

1

Extractive QA models often rely heavily on the quality and length of the input passage; too short or too long passages can reduce accuracy.

2

The choice of tokenizer and how text is split into tokens can affect the exact span predicted, impacting answer precision.

3

Fine-tuning on domain-specific data significantly improves performance, as general models may miss specialized terminology or phrasing.

When NOT to use

Extractive QA is not suitable when answers require synthesis, summarization, or reasoning beyond text spans. In such cases, generative QA or multi-hop reasoning models are better alternatives.

Production Patterns

In real systems, Extractive QA is combined with document retrieval to first find relevant passages, then extract answers. It is also used with confidence scoring to decide when to show answers or ask for human help.

Connections

Information Retrieval

Extractive QA builds on Information Retrieval by first locating relevant documents or passages before extracting answers.

Understanding retrieval helps grasp how QA systems narrow down large text collections to manageable chunks for answer extraction.

Span Prediction in NLP

Extractive QA is a specific application of span prediction, where models identify start and end positions in text.

Knowing span prediction techniques clarifies the core mechanism behind answer extraction.

Legal Document Review

Extractive QA techniques are used in legal tech to find exact clauses or facts in contracts and laws.

Seeing Extractive QA applied in law shows its power to speed up complex text analysis in real-world fields.

Common Pitfalls

#1 Expecting the model to generate answers not present in the text.

Wrong approach:

Question: 'Who invented the telephone?'
Passage: 'Alexander Graham Bell was a scientist.'
Model output: 'Thomas Edison' (made-up answer)

Correct approach:

Model should return 'no answer' or the exact text if present, e.g., 'Alexander Graham Bell' if mentioned.

Root cause:

Misunderstanding that Extractive QA only selects text, not generates new information.

#2 Feeding very long documents directly to the model without splitting.

Wrong approach:

Inputting entire book text as one passage for answer extraction.

Correct approach:

Split the document into smaller passages or paragraphs before running Extractive QA.

Root cause:

Ignoring model input length limits and performance degradation on long texts.

#3 Ignoring no-answer detection and always trusting model output.

Wrong approach:

Accepting any extracted span as correct answer even if irrelevant.

Correct approach:

Use models or thresholds that can indicate no-answer when appropriate.

Root cause:

Not accounting for cases where the passage lacks the answer.

Key Takeaways

Extractive QA finds answers by selecting exact text spans from a passage, not by creating new text.

Models predict start and end positions of answers using deep learning and contextual understanding.

Handling no-answer cases is essential for reliable real-world QA systems.

Training requires examples with questions, passages, and exact answer spans to learn effectively.

Extractive QA works best combined with document retrieval and careful passage selection.

Practice

1. What is the main goal of extractive question answering (QA)?

easy

A. To translate the question into another language

B. To generate a new answer not present in the text

C. To summarize the entire text into a short paragraph

D. To find the exact answer span within a given text

Solution

1. Step 1: Understand extractive QA purpose

   Extractive QA aims to locate the exact part of the text that answers the question.

2. Step 2: Compare options with definition

   Only To find the exact answer span within a given text describes finding the exact answer span inside the text, which matches extractive QA.

3. Final Answer:

   To find the exact answer span within a given text -> Option D

4. Quick Check:

   Extractive QA = find exact answer span [OK]

Hint: Extractive QA picks text parts, not creates new answers [OK]

Common Mistakes:

• Confusing extractive QA with generative QA
• Thinking extractive QA summarizes text
• Assuming extractive QA translates questions

2. Which of the following is the correct way to represent an extractive QA model's output?

easy

A. Span of text indices indicating the answer start and end

B. Single integer representing the answer length

C. List of unrelated keywords from the text

D. Boolean value indicating if the answer exists

Solution

1. Step 1: Recall extractive QA output format

   Extractive QA models output the start and end positions of the answer span in the text.

2. Step 2: Match options to output format

   Only Span of text indices indicating the answer start and end correctly describes output as text span indices.

3. Final Answer:

   Span of text indices indicating the answer start and end -> Option A

4. Quick Check:

   Output = start and end indices [OK]

Hint: Extractive QA outputs answer span positions, not just length [OK]

Common Mistakes:

• Choosing answer length instead of span indices
• Confusing keywords with answer span
• Thinking output is just true/false

3. Given the context: 'The Eiffel Tower is located in Paris.' and the question: 'Where is the Eiffel Tower?', what would an extractive QA model most likely output?

medium

A. "Eiffel Tower"

B. "located"

C. "Paris"

D. "The Eiffel Tower is located"

Solution

1. Step 1: Understand question and context

   The question asks for the location of the Eiffel Tower, which is stated as "Paris" in the context.

2. Step 2: Identify exact answer span

   The extractive QA model selects the exact text span answering the question, which is "Paris".

3. Final Answer:

   "Paris" -> Option C

4. Quick Check:

   Answer = "Paris" [OK]

Hint: Extractive QA picks exact answer phrase from context [OK]

Common Mistakes:

• Selecting part of the question as answer
• Choosing unrelated words from context
• Picking longer phrases than needed

4. Consider this extractive QA model output code snippet:

start_idx = 10
end_idx = 5
answer = context[start_idx:end_idx]

What is the main issue here?

medium

A. The end index is smaller than the start index, causing an empty answer

B. The indices are correct and will extract the answer properly

C. The code is missing a question input

D. The context variable is undefined

Solution

1. Step 1: Analyze index values

   The start index is 10 and the end index is 5, which is smaller than start.

2. Step 2: Understand slicing behavior

   In Python, slicing with start > end returns an empty string, so no answer is extracted.

3. Final Answer:

   The end index is smaller than the start index, causing an empty answer -> Option A

4. Quick Check:

   End index < start index = empty slice [OK]

Hint: End index must be >= start index for valid slice [OK]

Common Mistakes:

• Ignoring index order in slicing
• Assuming code runs without error
• Overlooking empty string result

5. You want to improve an extractive QA model to handle questions where the answer might not be present in the context. Which approach is best?

hard

A. Use a generative model instead of extractive QA

B. Add a 'no answer' prediction option so the model can say answer is missing

C. Train the model only on questions with guaranteed answers

D. Force the model to always select some text span regardless

Solution

1. Step 1: Understand the problem of missing answers

   Extractive QA models can fail if forced to select an answer when none exists in context.

2. Step 2: Evaluate solution options

   Adding a 'no answer' option lets the model explicitly indicate no answer is found, improving reliability.

3. Final Answer:

   Add a 'no answer' prediction option so the model can say answer is missing -> Option B

4. Quick Check:

   Handle missing answers = add 'no answer' option [OK]

Hint: Allow model to say 'no answer' when context lacks answer [OK]

Common Mistakes:

• Forcing answer selection even if none exists
• Ignoring questions without answers during training
• Switching to generative models unnecessarily