Open-domain QA basics in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to load a pre-trained question answering model using Hugging Face Transformers.

NLP

from transformers import pipeline
qa_pipeline = pipeline('[1]')

Drag options to blanks, or click blank then click option'

Aquestion-answering

Btext-generation

Csentiment-analysis

Dtranslation

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'text-generation' instead of 'question-answering'.

Confusing sentiment analysis with question answering.

The pipeline type for open-domain question answering is 'question-answering'.

2fill in blank

medium

Complete the code to get the answer from the QA pipeline given a question and context.

NLP

result = qa_pipeline({'question': '[1]', 'context': 'The Eiffel Tower is in Paris.'})

Drag options to blanks, or click blank then click option'

AWhen was the Eiffel Tower built?

BWhat is the capital of France?

CWhere is the Eiffel Tower?

DWho built the Eiffel Tower?

Attempts:

3 left

💡 Hint

Common Mistakes

Asking about the builder or date which is not in the context.

Using a question unrelated to the context.

The question 'Where is the Eiffel Tower?' matches the context about the Eiffel Tower's location.

3fill in blank

hard

Fix the error in the code to extract the answer text from the result dictionary.

NLP

answer_text = result['[1]']

Drag options to blanks, or click blank then click option'

Atext

Banswer

Cresponse

Dresult

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'text' or 'response' which are not keys in the result.

Trying to access 'result' key inside the result dictionary.

The key 'answer' contains the predicted answer text in the result dictionary.

4fill in blank

hard

Fill both blanks to create a dictionary with question and context for the QA pipeline.

NLP

input_data = {'[1]': 'What is AI?', '[2]': 'AI stands for Artificial Intelligence.'}

Drag options to blanks, or click blank then click option'

Aquestion

Banswer

Ccontext

Dresponse

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'answer' or 'response' as keys instead of 'question' and 'context'.

Mixing up the order of keys.

The QA pipeline expects keys 'question' and 'context' in the input dictionary.

5fill in blank

hard

Fill all three blanks to run the QA pipeline and print the answer.

NLP

result = qa_pipeline({'[1]': '[2]', '[3]': 'Python is a programming language.'})
print(result['answer'])

Drag options to blanks, or click blank then click option'

Aquestion

BWhat is Python?

Ccontext

Danswer

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'answer' as a key in the input dictionary.

Putting the question in the 'context' key.

The input dictionary keys are 'question' and 'context', and the question is 'What is Python?'.

Practice

1. What is the main goal of open-domain question answering (QA)?

easy

A. To summarize a single document

B. To translate text from one language to another

C. To find answers to any question from a large collection of texts

D. To generate new text based on a prompt

Solution

1. Step 1: Understand the definition of open-domain QA

   Open-domain QA aims to answer questions using a wide range of texts, not limited to a specific topic.

2. Step 2: Compare options with this definition

   Only To find answers to any question from a large collection of texts matches this goal; others describe different NLP tasks.

3. Final Answer:

   To find answers to any question from a large collection of texts -> Option C

4. Quick Check:

   Open-domain QA = Finding answers from many texts [OK]

Hint: Open-domain QA means answering questions from many texts [OK]

Common Mistakes:

• Confusing QA with translation
• Thinking QA only summarizes text
• Mixing QA with text generation

2. Which of the following is the correct sequence of steps in an open-domain QA system?

easy

A. Classify questions, then ignore documents

B. Generate answers first, then find documents

C. Summarize documents, then translate answers

D. Retrieve relevant documents, then read and extract answers

Solution

1. Step 1: Recall the typical open-domain QA pipeline

   It first retrieves relevant documents, then reads them to find answers.

2. Step 2: Match options to this pipeline

   Only Retrieve relevant documents, then read and extract answers correctly describes this order; others are incorrect or unrelated.

3. Final Answer:

   Retrieve relevant documents, then read and extract answers -> Option D

4. Quick Check:

   QA steps = Retrieve then read [OK]

Hint: QA first finds texts, then reads for answers [OK]

Common Mistakes:

• Thinking answer generation happens before retrieval
• Confusing summarization with QA
• Ignoring the retrieval step

3. Given this Python snippet using a pretrained QA model:

from transformers import pipeline
qa = pipeline('question-answering')
context = "The Eiffel Tower is in Paris."
question = "Where is the Eiffel Tower located?"
result = qa(question=question, context=context)
print(result['answer'])

What will be printed?

medium

A. Paris

B. Eiffel Tower

C. question-answering

D. The Eiffel Tower

Solution

1. Step 1: Understand the QA pipeline usage

   The pipeline takes a question and context, then returns the answer span from the context.

2. Step 2: Identify the answer span in the context

   The question asks for location; context says "The Eiffel Tower is in Paris." The answer is "Paris".

3. Final Answer:

   Paris -> Option A

4. Quick Check:

   Answer extracted = Paris [OK]

Hint: QA model returns the answer span from context [OK]

Common Mistakes:

• Printing the question instead of answer
• Confusing the model name with output
• Selecting the full sentence instead of answer span

4. You have this code snippet for open-domain QA:

from transformers import pipeline
qa = pipeline('question-answering')
context = "Mount Everest is the highest mountain."
question = "What is the highest mountain?"
result = qa(question=question, context=context)
print(result['answer'])

But it raises a KeyError: 'answer'. What is the likely cause?

medium

A. The context is empty

B. The pipeline was not properly initialized for question-answering

C. The question is not a string

D. The print statement is incorrect

Solution

1. Step 1: Analyze the error KeyError: 'answer'

   This error means the result dictionary does not have the key 'answer'.

2. Step 2: Check pipeline initialization

   If the pipeline is not correctly set for 'question-answering', the output format differs and lacks 'answer'.

3. Final Answer:

   The pipeline was not properly initialized for question-answering -> Option B

4. Quick Check:

   Wrong pipeline type causes missing 'answer' key [OK]

Hint: Ensure pipeline type matches task to get correct keys [OK]

Common Mistakes:

• Assuming context is empty without checking
• Ignoring pipeline initialization errors
• Misreading error as print statement issue

5. You want to improve an open-domain QA system that sometimes returns wrong answers because it reads irrelevant documents. Which approach helps most?

hard

A. Improve the document retrieval step to find more relevant texts

B. Use a smaller pretrained model to speed up reading

C. Remove the retrieval step and read all documents

D. Translate questions to another language before answering

Solution

1. Step 1: Identify the problem cause

   Wrong answers happen because the system reads irrelevant documents.

2. Step 2: Choose the best fix

   Improving retrieval to get relevant documents reduces wrong answers effectively.

3. Final Answer:

   Improve the document retrieval step to find more relevant texts -> Option A

4. Quick Check:

   Better retrieval = better answer relevance [OK]

Hint: Better retrieval means better answers [OK]

Common Mistakes:

• Thinking smaller models improve accuracy
• Removing retrieval causes overload and noise
• Translating questions doesn't fix relevance