Extractive QA concept in NLP - Deep Dive

Overview - Extractive QA concept

What is it?

Extractive Question Answering (QA) is a method where a system finds the exact answer to a question by selecting a piece of text from a given document or passage. Instead of generating new text, it extracts the answer directly from the source. This helps machines understand and respond to questions using existing information.

Why it matters

Extractive QA solves the problem of quickly finding precise answers from large amounts of text, like articles or reports. Without it, people would have to read everything themselves, which is slow and tiring. It powers search engines, virtual assistants, and customer support by giving fast, accurate answers.

Where it fits

Before learning Extractive QA, you should understand basic natural language processing concepts like tokenization and embeddings. After this, you can explore generative QA, where answers are created rather than extracted, and advanced models like transformers for better understanding.

Mental Model

Core Idea

Extractive QA works by scanning a text to find the exact part that answers a question, like highlighting a sentence in a book.

Think of it like...

Imagine you have a big book and someone asks you a question. Instead of rewriting the answer, you flip through the pages and point to the exact sentence that answers them.

┌───────────────┐
│   Question    │
└──────┬────────┘
       │
       ▼
┌─────────────────────────────┐
│       Text Passage           │
│  ┌───────────────────────┐  │
│  │  Extracted Answer     │  │
│  │  (Exact text snippet) │  │
│  └───────────────────────┘  │
└─────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Questions and Text

Concept: Learn what questions and text passages are in simple terms and how they relate.

A question is something you want to know, like 'What is the capital of France?'. A text passage is a piece of writing that might contain the answer, like a paragraph about France. Extractive QA finds the answer inside this passage.

Result

You can identify a question and a text passage that might contain the answer.

Understanding the basic elements of question and text is essential before trying to find answers inside text.

2

Foundation

What Does Extractive Mean?

Concept: Extractive means taking out something exactly as it is, without changing it.

In Extractive QA, the system does not create new sentences. It picks the exact words or sentences from the text that answer the question. This is different from making up an answer.

Result

You know that extractive means copying the answer directly from the text.

Knowing that answers come directly from the text helps set expectations for how the system works.

3

Intermediate

How Models Find Answers in Text

🤔Before reading on: do you think the model reads the whole text at once or word by word? Commit to your answer.

Concept: Models scan the entire text passage to find the start and end positions of the answer span.

Modern Extractive QA models look at the whole passage and the question together. They predict where the answer starts and ends by assigning scores to each word or token. The highest scoring span is chosen as the answer.

Result

The model outputs the exact snippet from the text that answers the question.

Understanding that models predict answer boundaries explains why they can find precise answers instead of guessing.

4

Intermediate

Role of Contextual Word Representations

🤔Before reading on: do you think the model treats each word the same or considers surrounding words? Commit to your answer.

Concept: Contextual word representations help the model understand words based on their surrounding words.

Words can have different meanings depending on context. For example, 'bank' can mean river edge or money place. Models use techniques like transformers to create word vectors that change meaning based on context, helping find the right answer.

Result

The model better understands the question and passage, improving answer accuracy.

Knowing that word meaning depends on context is key to why modern QA models work well.

5

Intermediate

Training Extractive QA Models

🤔Before reading on: do you think models learn from questions only or from questions with answers? Commit to your answer.

Concept: Models learn by seeing many examples of questions paired with exact answer spans in text.

During training, the model is given a question, a passage, and the correct answer span. It adjusts itself to predict the right start and end positions. Over time, it gets better at spotting answers in new texts.

Result

The model can generalize to find answers in unseen passages.

Understanding training with answer spans clarifies how models improve and why labeled data is important.

6

Advanced

Handling No-Answer Questions

🤔Before reading on: do you think every question always has an answer in the text? Commit to your answer.

Concept: Some questions may not have an answer in the given passage, and models must detect this.

Advanced Extractive QA models include a special option to say 'no answer' if the passage doesn't contain the answer. This prevents wrong guesses and improves reliability.

Result

The model can correctly say when no answer is found, avoiding misleading outputs.

Knowing how models handle no-answer cases is crucial for real-world applications where not all questions are answerable.

7

Expert

Limitations and Biases in Extractive QA

🤔Before reading on: do you think Extractive QA models always find the correct answer if it is in the text? Commit to your answer.

Concept: Extractive QA models can fail due to ambiguous questions, biased training data, or complex language.

Models sometimes pick wrong spans if the question is unclear or the text is tricky. They also reflect biases from training data, like favoring common answers. Understanding these limits helps improve and trust models.

Result

You recognize when and why models might make mistakes.

Awareness of model limitations guides better use and development of Extractive QA systems.

Under the Hood

Extractive QA models use deep neural networks, often transformers, to encode both the question and passage into vectors. They then compute scores for each token's likelihood of being the start or end of the answer span. The highest scoring span is selected as the answer. During training, the model learns to assign high scores to correct spans by minimizing a loss function comparing predictions to true answers.

Why designed this way?

Extractive QA was designed to provide precise answers without generating new text, reducing errors and hallucinations common in generative models. Using start and end positions simplifies the problem to span prediction, which is easier to train and interpret. Transformers were chosen for their ability to capture context and relationships in text effectively.

┌───────────────┐      ┌───────────────┐
│   Question    │      │  Text Passage  │
└──────┬────────┘      └──────┬────────┘
       │                      │
       │                      │
       ▼                      ▼
  ┌─────────────────────────────────┐
  │      Transformer Encoder         │
  │  (jointly encodes question &    │
  │   passage into contextual tokens)│
  └──────────────┬──────────────────┘
                 │
                 ▼
       ┌─────────────────────┐
       │ Start & End Scorers  │
       │ (predict answer span)│
       └─────────┬───────────┘
                 │
                 ▼
       ┌─────────────────────┐
       │ Extracted Answer Span│
       └─────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does Extractive QA generate new sentences as answers? Commit to yes or no.

Common Belief: Extractive QA creates new answers by writing sentences based on the question.

Reality: Extractive QA only selects exact text spans from the given passage; it does not generate new text.

Why it matters: Believing it generates text can lead to expecting creative or summarized answers, causing confusion when only exact text is returned.

Quick: Do you think Extractive QA always finds the correct answer if it exists in the text? Commit to yes or no.

Common Belief: If the answer is in the text, the model will always find it correctly.

Reality: Models can make mistakes due to ambiguous questions, complex language, or insufficient training data.

Why it matters: Overtrusting models can cause users to accept wrong answers, leading to misinformation or errors.

Quick: Does Extractive QA require the entire document to answer a question? Commit to yes or no.

Common Belief: Extractive QA models need the full document to find answers.

Reality: Models usually work on smaller passages or chunks because processing very long texts is inefficient and less accurate.

Why it matters: Expecting full-document processing can cause performance issues and misunderstandings about model capabilities.

Quick: Can Extractive QA handle questions with no answers in the text? Commit to yes or no.

Common Belief: Extractive QA always returns an answer, even if the text doesn't contain it.

Reality: Advanced models can detect no-answer cases and return 'no answer' instead of guessing.

Why it matters: Knowing this prevents accepting false answers when the text lacks relevant information.

Expert Zone

1

Extractive QA models often rely heavily on the quality and length of the input passage; too short or too long passages can reduce accuracy.

2

The choice of tokenizer and how text is split into tokens can affect the exact span predicted, impacting answer precision.

3

Fine-tuning on domain-specific data significantly improves performance, as general models may miss specialized terminology or phrasing.

When NOT to use

Extractive QA is not suitable when answers require synthesis, summarization, or reasoning beyond text spans. In such cases, generative QA or multi-hop reasoning models are better alternatives.

Production Patterns

In real systems, Extractive QA is combined with document retrieval to first find relevant passages, then extract answers. It is also used with confidence scoring to decide when to show answers or ask for human help.

Connections

Information Retrieval

Extractive QA builds on Information Retrieval by first locating relevant documents or passages before extracting answers.

Understanding retrieval helps grasp how QA systems narrow down large text collections to manageable chunks for answer extraction.

Span Prediction in NLP

Extractive QA is a specific application of span prediction, where models identify start and end positions in text.

Knowing span prediction techniques clarifies the core mechanism behind answer extraction.

Legal Document Review

Extractive QA techniques are used in legal tech to find exact clauses or facts in contracts and laws.

Seeing Extractive QA applied in law shows its power to speed up complex text analysis in real-world fields.

Common Pitfalls

#1 Expecting the model to generate answers not present in the text.

Wrong approach:

Question: 'Who invented the telephone?'
Passage: 'Alexander Graham Bell was a scientist.'
Model output: 'Thomas Edison' (made-up answer)

Correct approach:

Model should return 'no answer' or the exact text if present, e.g., 'Alexander Graham Bell' if mentioned.

Root cause:

Misunderstanding that Extractive QA only selects text, not generates new information.

#2 Feeding very long documents directly to the model without splitting.

Wrong approach:

Inputting entire book text as one passage for answer extraction.

Correct approach:

Split the document into smaller passages or paragraphs before running Extractive QA.

Root cause:

Ignoring model input length limits and performance degradation on long texts.

#3 Ignoring no-answer detection and always trusting model output.

Wrong approach:

Accepting any extracted span as correct answer even if irrelevant.

Correct approach:

Use models or thresholds that can indicate no-answer when appropriate.

Root cause:

Not accounting for cases where the passage lacks the answer.

Key Takeaways

Extractive QA finds answers by selecting exact text spans from a passage, not by creating new text.

Models predict start and end positions of answers using deep learning and contextual understanding.

Handling no-answer cases is essential for reliable real-world QA systems.

Training requires examples with questions, passages, and exact answer spans to learn effectively.

Extractive QA works best combined with document retrieval and careful passage selection.