Extractive QA concept in NLP - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Extractive Question Answering (QA)?

Extractive QA is a type of question answering where the answer is found by selecting a span of text directly from a given passage or document.

beginner

How does Extractive QA differ from Generative QA?

Extractive QA picks answers directly from the text, while Generative QA creates answers in its own words, possibly not found exactly in the text.

beginner

What is the typical input for an Extractive QA model?

The input usually includes a question and a context passage where the answer is expected to be found.

intermediate

Which machine learning models are commonly used for Extractive QA?

Models like BERT and its variants are commonly used because they understand context well and can find answer spans in text.

intermediate

What are start and end tokens in Extractive QA?

They are positions in the text that mark where the answer begins and ends, helping the model select the exact answer span.

In Extractive QA, where does the answer come from?

A From user input

B Generated from scratch by the model

C Directly from the given text passage

D From a database lookup

Extractive QA selects the answer span directly from the provided text passage.

What does an Extractive QA model predict to find the answer?

A A generated sentence unrelated to the passage

B A summary of the passage

C A yes or no answer

D Start and end positions of the answer in the text

The model predicts the start and end tokens to extract the answer span.

Which of these is a popular model architecture for Extractive QA?

A K-Means

B BERT

C GAN

D RNN without attention

BERT is widely used for Extractive QA due to its strong contextual understanding.

What is the main difference between Extractive and Generative QA?

A Extractive selects text from passage; Generative creates new text

B Extractive creates new text; Generative selects from passage

C Both generate new text

D Both select text from passage

Extractive QA picks answers from the passage, Generative QA creates answers.

What inputs does an Extractive QA system need?

A A question and a context passage

B Only a question

C Only a passage

D A question and an answer

Extractive QA requires both a question and a passage to find the answer.

Explain how an Extractive QA model finds an answer in a text passage.

Think about how the model points to the exact words in the text.

Describe the difference between Extractive and Generative QA in simple terms.

Consider whether the answer is copied or made up.

Practice

1. What is the main goal of extractive question answering (QA)?

easy

A. To translate the question into another language

B. To generate a new answer not present in the text

C. To summarize the entire text into a short paragraph

D. To find the exact answer span within a given text

Solution

1. Step 1: Understand extractive QA purpose

   Extractive QA aims to locate the exact part of the text that answers the question.

2. Step 2: Compare options with definition

   Only To find the exact answer span within a given text describes finding the exact answer span inside the text, which matches extractive QA.

3. Final Answer:

   To find the exact answer span within a given text -> Option D

4. Quick Check:

   Extractive QA = find exact answer span [OK]

Hint: Extractive QA picks text parts, not creates new answers [OK]

Common Mistakes:

• Confusing extractive QA with generative QA
• Thinking extractive QA summarizes text
• Assuming extractive QA translates questions

2. Which of the following is the correct way to represent an extractive QA model's output?

easy

A. Span of text indices indicating the answer start and end

B. Single integer representing the answer length

C. List of unrelated keywords from the text

D. Boolean value indicating if the answer exists

Solution

1. Step 1: Recall extractive QA output format

   Extractive QA models output the start and end positions of the answer span in the text.

2. Step 2: Match options to output format

   Only Span of text indices indicating the answer start and end correctly describes output as text span indices.

3. Final Answer:

   Span of text indices indicating the answer start and end -> Option A

4. Quick Check:

   Output = start and end indices [OK]

Hint: Extractive QA outputs answer span positions, not just length [OK]

Common Mistakes:

• Choosing answer length instead of span indices
• Confusing keywords with answer span
• Thinking output is just true/false

3. Given the context: 'The Eiffel Tower is located in Paris.' and the question: 'Where is the Eiffel Tower?', what would an extractive QA model most likely output?

medium

A. "Eiffel Tower"

B. "located"

C. "Paris"

D. "The Eiffel Tower is located"

Solution

1. Step 1: Understand question and context

   The question asks for the location of the Eiffel Tower, which is stated as "Paris" in the context.

2. Step 2: Identify exact answer span

   The extractive QA model selects the exact text span answering the question, which is "Paris".

3. Final Answer:

   "Paris" -> Option C

4. Quick Check:

   Answer = "Paris" [OK]

Hint: Extractive QA picks exact answer phrase from context [OK]

Common Mistakes:

• Selecting part of the question as answer
• Choosing unrelated words from context
• Picking longer phrases than needed

4. Consider this extractive QA model output code snippet:

start_idx = 10
end_idx = 5
answer = context[start_idx:end_idx]

What is the main issue here?

medium

A. The end index is smaller than the start index, causing an empty answer

B. The indices are correct and will extract the answer properly

C. The code is missing a question input

D. The context variable is undefined

Solution

1. Step 1: Analyze index values

   The start index is 10 and the end index is 5, which is smaller than start.

2. Step 2: Understand slicing behavior

   In Python, slicing with start > end returns an empty string, so no answer is extracted.

3. Final Answer:

   The end index is smaller than the start index, causing an empty answer -> Option A

4. Quick Check:

   End index < start index = empty slice [OK]

Hint: End index must be >= start index for valid slice [OK]

Common Mistakes:

• Ignoring index order in slicing
• Assuming code runs without error
• Overlooking empty string result

5. You want to improve an extractive QA model to handle questions where the answer might not be present in the context. Which approach is best?

hard

A. Use a generative model instead of extractive QA

B. Add a 'no answer' prediction option so the model can say answer is missing

C. Train the model only on questions with guaranteed answers

D. Force the model to always select some text span regardless

Solution

1. Step 1: Understand the problem of missing answers

   Extractive QA models can fail if forced to select an answer when none exists in context.

2. Step 2: Evaluate solution options

   Adding a 'no answer' option lets the model explicitly indicate no answer is found, improving reliability.

3. Final Answer:

   Add a 'no answer' prediction option so the model can say answer is missing -> Option B

4. Quick Check:

   Handle missing answers = add 'no answer' option [OK]

Hint: Allow model to say 'no answer' when context lacks answer [OK]

Common Mistakes:

• Forcing answer selection even if none exists
• Ignoring questions without answers during training
• Switching to generative models unnecessarily