Question answering in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of a question answering system?

Choose the best description of what a question answering system does.

A It predicts numerical values based on historical data.

B It generates answers to questions by understanding and processing natural language input.

C It translates text from one language to another without understanding the meaning.

D It classifies images into categories based on their content.

💡 Hint

Think about what happens when you ask a smart assistant a question.

Question answering systems are designed to understand questions and provide relevant answers by processing natural language.

❓ Predict Output

intermediate

Output of a simple QA model prediction

Given the following code snippet using a pretrained QA model, what will be the printed answer?

Prompt Engineering / GenAI

from transformers import pipeline
qa = pipeline('question-answering')
context = "The Eiffel Tower is located in Paris."
question = "Where is the Eiffel Tower located?"
result = qa(question=question, context=context)
print(result['answer'])

A "located"

B "Eiffel Tower"

C "The Eiffel Tower"

D "Paris"

💡 Hint

The model extracts the answer from the context that best fits the question.

The model finds the phrase in the context that answers the question, which is "Paris".

❓ Model Choice

advanced

Choosing the best model type for open-domain QA

You want to build a system that answers questions about any topic using a large collection of documents. Which model type is best suited?

A A retrieval-augmented generation model that first finds relevant documents then generates answers.

B A simple text classification model that labels questions into categories.

C A convolutional neural network trained on images.

D A clustering algorithm that groups similar questions.

💡 Hint

Think about how to handle large knowledge bases and generate answers.

Retrieval-augmented generation models combine document search with answer generation, ideal for open-domain QA.

❓ Metrics

advanced

Evaluating a QA model with Exact Match and F1 score

A QA model predicted the answer "Paris" for the question "Where is the Eiffel Tower?" The true answer is "Paris, France". What are the Exact Match and F1 scores?

A Exact Match: 0, F1: 0.67

B Exact Match: 1, F1: 1

C Exact Match: 0, F1: 0

D Exact Match: 1, F1: 0.5

💡 Hint

Exact Match requires exact string equality; F1 measures overlap of tokens.

Exact Match is 0 because "Paris" != "Paris, France" exactly. F1 is calculated based on token overlap: tokens in common are "Paris" (1), total tokens predicted (1), total tokens true (2), so F1 = 2*1/(1+2) = 0.67.

🔧 Debug

expert

Why does this QA model code raise an error?

Consider this code snippet:

from transformers import pipeline
qa = pipeline('question-answering')
context = None
question = "What is AI?"
result = qa(question=question, context=context)
print(result['answer'])

Why does it raise an error?

A Because the pipeline is not initialized, it raises an ImportError.

B Because the question is missing, the model raises a ValueError.

C Because the context is None, the model cannot process it and raises a TypeError.

D Because the print statement is incorrect syntax, it raises a SyntaxError.

💡 Hint

Check the type and value of the context variable passed to the model.

The context must be a string. Passing None causes the model to fail with a TypeError because it cannot process a NoneType input.