Abstractive summarization in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main difference between extractive and abstractive summarization?

Choose the option that best describes how abstractive summarization differs from extractive summarization.

A Abstractive summarization copies sentences verbatim, while extractive summarization paraphrases the content.

B Abstractive summarization only selects the longest sentences, while extractive summarization selects the shortest sentences.

C Abstractive summarization generates new sentences that capture the meaning, while extractive summarization selects existing sentences from the text.

D Abstractive summarization uses keyword matching, while extractive summarization uses neural networks.

💡 Hint

Think about whether the summary is created by copying or by generating new text.

Abstractive summarization creates new sentences that may not appear in the original text, capturing the core meaning. Extractive summarization picks sentences directly from the text.

❓ Predict Output

intermediate

Output of a simple abstractive summarization model

Given the following Python code using Hugging Face's transformers library, what is the output summary?

NLP

from transformers import pipeline
summarizer = pipeline('summarization')
text = "The Eiffel Tower is a wrought-iron lattice tower on the Champ de Mars in Paris, France. It is named after the engineer Gustave Eiffel, whose company designed and built the tower. The tower is 324 meters tall and was completed in 1889."
summary = summarizer(text, max_length=30, min_length=10, do_sample=False)
print(summary[0]['summary_text'])

A The Eiffel Tower is a 324-meter tall wrought-iron lattice tower in Paris, built by Gustave Eiffel's company and completed in 1889.

B The Eiffel Tower is a famous tower in France named after Gustave Eiffel.

C The Eiffel Tower was built in 1889 and is located in Paris.

D The Eiffel Tower is a tall tower built by Gustave Eiffel.

💡 Hint

Look for the most detailed and complete summary that fits the length constraints.

The summarizer produces a concise summary that includes key facts: height, location, builder, and completion year, matching option A.

❓ Hyperparameter

advanced

Effect of changing max_length in abstractive summarization

In an abstractive summarization model, what is the effect of increasing the max_length parameter during generation?

A The model will generate summaries faster.

B The summary will be shorter and more concise.

C The model will ignore the input text and generate random text.

D The summary will generally be longer and may include more details.

💡 Hint

Think about what controlling the maximum length of output text means for the summary size.

Increasing max_length allows the model to produce longer summaries, potentially including more information.

❓ Metrics

advanced

Which metric is most suitable for evaluating abstractive summarization quality?

Choose the metric that best measures how well an abstractive summarization model captures the meaning of the original text.

A Mean Squared Error, measuring numeric differences between summary and original text.

B ROUGE score, which compares overlap of n-grams between generated and reference summaries.

C BLEU score, which is only used for machine translation.

D Accuracy, measuring exact word matches between summary and original text.

💡 Hint

Think about a metric that compares summaries based on overlapping phrases.

ROUGE is widely used for summarization evaluation because it measures overlap of words and phrases between generated and reference summaries.

🔧 Debug

expert

Why does this abstractive summarization model output repetitive text?

Consider this code snippet using a pretrained summarization model:

from transformers import pipeline
summarizer = pipeline('summarization')
text = "Deep learning models are powerful. Deep learning models are powerful. Deep learning models are powerful."
summary = summarizer(text, max_length=20, min_length=5, do_sample=False)
print(summary[0]['summary_text'])

The output is: "Deep learning models are powerful. Deep learning models are powerful. Deep learning models are powerful." What is the most likely cause?

A The model is overfitting and repeating phrases because of lack of diversity in generation settings.

B The input text is too short, so the model repeats it to fill the summary length.

C The model is not pretrained for summarization and outputs input text verbatim.

D The max_length parameter is too large, causing repetition.

💡 Hint

Consider how generation parameters affect repetition in output.

Using do_sample=False with greedy decoding can cause repetitive outputs, especially on repetitive input. This is a known issue with lack of diversity in generation.