QA with Hugging Face pipeline in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple QA pipeline

What is the output of this code snippet using Hugging Face's QA pipeline?

NLP

from transformers import pipeline
qa = pipeline('question-answering')
context = "The Eiffel Tower is located in Paris."
question = "Where is the Eiffel Tower located?"
result = qa(question=question, context=context)
print(result['answer'])

A "Paris"

B "Eiffel Tower"

C "located"

D "The Eiffel Tower"

💡 Hint

The pipeline extracts the answer span from the context that best answers the question.

The QA pipeline finds the answer span in the context. Since the question asks where the Eiffel Tower is located, the answer is "Paris".

❓ Model Choice

intermediate

Choosing the right model for QA pipeline

Which model is best suited for a question-answering pipeline that requires understanding context and providing precise answers?

A "bert-base-uncased"

B "roberta-base"

C "distilbert-base-uncased-distilled-squad"

D "gpt2"

💡 Hint

Look for a model fine-tuned specifically on a QA dataset like SQuAD.

The 'distilbert-base-uncased-distilled-squad' model is fine-tuned on the SQuAD dataset, making it suitable for QA tasks.

❓ Hyperparameter

advanced

Effect of changing top_k in QA pipeline

In the Hugging Face QA pipeline, what happens if you set the parameter top_k=3 when calling the pipeline?

A The pipeline returns the top 3 most probable answers instead of just one.

B The pipeline limits the context to the first 3 sentences.

C The pipeline uses only 3 tokens from the question for answering.

D The pipeline runs 3 times and returns the last answer.

💡 Hint

top_k controls how many answers the model returns.

Setting top_k=3 makes the pipeline return the three best answers ranked by confidence.

❓ Metrics

advanced

Evaluating QA model performance

Which metric is commonly used to evaluate the accuracy of a question-answering model on datasets like SQuAD?

A BLEU score

B Exact Match (EM) score

C Mean Squared Error (MSE)

D Perplexity

💡 Hint

This metric measures if the predicted answer exactly matches the ground truth.

Exact Match (EM) measures the percentage of predictions that exactly match the correct answers.

🔧 Debug

expert

Debugging a QA pipeline error

You run this code but get a TypeError: 'NoneType' object is not subscriptable. What is the cause? from transformers import pipeline qa = pipeline('question-answering') context = None question = "What is AI?" result = qa(question=question, context=context) print(result['answer'])

A The model does not support question-answering.

B The question variable is None, causing the error.

C The pipeline is not initialized correctly.

D The context variable is None, so the pipeline cannot find an answer.

💡 Hint

Check the input types passed to the pipeline.

The context is None, so the pipeline returns None, causing the TypeError when trying to access ['answer'].