Extractive summarization in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the library used for extractive summarization.

NLP

from sklearn.feature_extraction.text import [1]

Drag options to blanks, or click blank then click option'

ADictVectorizer

BCountVectorizer

CHashingVectorizer

DTfidfVectorizer

Attempts:

3 left

💡 Hint

Common Mistakes

Choosing CountVectorizer which only counts word occurrences without weighting.

Using HashingVectorizer which is less interpretable for summarization.

The TfidfVectorizer converts text to a matrix of TF-IDF features, which is commonly used in extractive summarization.

2fill in blank

medium

Complete the code to split the text into sentences for summarization.

NLP

import nltk
nltk.download('punkt')
sentences = nltk.tokenize.[1](text)

Drag options to blanks, or click blank then click option'

Aword_tokenize

Bsent_tokenize

Cregexp_tokenize

Dtweet_tokenize

Attempts:

3 left

💡 Hint

Common Mistakes

Using word_tokenize which splits text into words, not sentences.

Forgetting to download the 'punkt' package.

sent_tokenize splits text into sentences, which is essential for extractive summarization.

3fill in blank

hard

Fix the error in the code to compute cosine similarity matrix for sentence vectors.

NLP

from sklearn.metrics.pairwise import [1]
similarity_matrix = cosine_similarity(sentence_vectors)

Drag options to blanks, or click blank then click option'

Acosine_similarity

Beuclidean_distances

Cmanhattan_distances

Dpairwise_distances

Attempts:

3 left

💡 Hint

Common Mistakes

Using distance functions which measure dissimilarity, not similarity.

Importing a function but calling a different one.

cosine_similarity computes similarity between vectors, which is used to find sentence similarity in extractive summarization.

4fill in blank

hard

Fill both blanks to rank sentences using PageRank algorithm.

NLP

import networkx as nx
sentence_graph = nx.[1](similarity_matrix)
scores = nx.[2](sentence_graph)

Drag options to blanks, or click blank then click option'

Afrom_numpy_array

Bpagerank

Cdegree_centrality

Dto_numpy_array

Attempts:

3 left

💡 Hint

Common Mistakes

Using degree_centrality which is simpler but less effective for ranking.

Confusing graph creation and ranking functions.

We create a graph from the similarity matrix using from_numpy_array and rank sentences with pagerank.

5fill in blank

hard

Fill all three blanks to select top sentences and join them as summary.

NLP

top_sentences = sorted(((scores[i], s) for i, s in enumerate(sentences)), reverse=True)[:[1]]
summary = ' '.join([[2] for _, [3] in top_sentences])

Drag options to blanks, or click blank then click option'

A3

Bsentence

Cs

Dscore

Attempts:

3 left

💡 Hint

Common Mistakes

Joining scores instead of sentences.

Selecting too many or too few sentences.

We select top 3 sentences, then join the sentence strings (variable s) to form the summary.

Practice

1. What is the main goal of extractive summarization in NLP?

easy

A. To translate the text into another language

B. To rewrite the text using simpler words

C. To select important sentences from the original text to create a summary

D. To generate new sentences that explain the text

Solution

1. Step 1: Understand extractive summarization

   Extractive summarization picks key sentences directly from the original text without changing them.

2. Step 2: Compare options

   Only To select important sentences from the original text to create a summary describes selecting important sentences from the original text, which matches extractive summarization.

3. Final Answer:

   To select important sentences from the original text to create a summary -> Option C

4. Quick Check:

   Extractive summarization = selecting key sentences [OK]

Hint: Extractive means picking from original text directly [OK]

Common Mistakes:

• Confusing extractive with abstractive summarization
• Thinking it rewrites or translates text
• Assuming it generates new sentences

2. Which of the following is a common technique used in extractive summarization?

easy

A. Neural machine translation

B. Text generation with GPT

C. Part-of-speech tagging

D. TF-IDF scoring of sentences

Solution

1. Step 1: Identify techniques for extractive summarization

   Extractive summarization often uses TF-IDF to score sentences by importance based on word frequency.

2. Step 2: Eliminate unrelated options

   Neural machine translation and text generation are for other NLP tasks, and POS tagging is not directly used for summarization scoring.

3. Final Answer:

   TF-IDF scoring of sentences -> Option D

4. Quick Check:

   TF-IDF = common extractive technique [OK]

Hint: TF-IDF ranks sentence importance in extractive summarization [OK]

Common Mistakes:

• Confusing summarization with translation or generation
• Thinking POS tagging directly creates summaries
• Ignoring TF-IDF's role in scoring

3. Given the following Python code snippet using TF-IDF for extractive summarization, what will be the output?

from sklearn.feature_extraction.text import TfidfVectorizer

texts = ["Cats are great pets.", "Dogs are loyal animals.", "Cats and dogs can live together."]
vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(texts)
scores = X.sum(axis=1)
print(scores)

medium

A. [[0.0], [0.0], [0.0]]

B. [[2.0], [2.0], [2.4]]

C. [[2.0], [2.0], [3.0]]

D. [[1.0], [1.0], [1.0]]

Solution

1. Step 1: Understand TF-IDF vectorization and summing

   The code vectorizes three sentences and sums TF-IDF scores per sentence (row-wise sum).

2. Step 2: Calculate approximate sums

   Each sentence has TF-IDF scores summing roughly to 2.0, 2.0, and 2.4 respectively due to shared and unique words.

3. Final Answer:

   [[2.0], [2.0], [2.4]] -> Option B

4. Quick Check:

   Sum TF-IDF per sentence ≈ [[2.0], [2.0], [2.4]] [OK]

Hint: Sum TF-IDF scores per sentence to get importance [OK]

Common Mistakes:

• Assuming zero scores for all sentences
• Confusing sum with average
• Misunderstanding TF-IDF output shape

4. You have this extractive summarization code snippet:

sentences = ["AI is fascinating.", "It helps solve problems.", "AI can learn from data."]
scores = [0.8, 0.9, 0.85]
summary = []
for i in range(len(sentences)):
    if scores[i] > 0.85:
        summary.append(sentences[i])
print(summary)

What is the output and is there any bug?

medium

A. ['It helps solve problems.'] with no bug

B. ['AI is fascinating.', 'It helps solve problems.', 'AI can learn from data.'] with no bug

C. ['It helps solve problems.', 'AI can learn from data.'] but index error bug

D. [] because scores are not compared correctly

Solution

1. Step 1: Check score filtering condition

   The code adds sentences with scores > 0.85, so sentences with 0.9 and 0.85 are checked; 0.85 is not > 0.85, so only 0.9 and 0.85 fail or pass accordingly.

2. Step 2: Determine which sentences are included

   Scores: 0.8 (no), 0.9 (yes), 0.85 (no). So only "It helps solve problems." is included. But 0.85 is not > 0.85, so excluded.

3. Final Answer:

   ['It helps solve problems.'] -> Option A

4. Quick Check:

   Scores > 0.85 filter sentences correctly [OK]

Hint: Check strict > vs >= in score filtering [OK]

Common Mistakes:

• Including sentences with score equal to threshold
• Expecting index errors where none exist
• Misreading the comparison operator

5. You want to create an extractive summarizer that picks the top 2 sentences from a document based on TF-IDF scores. Given these sentences and their scores:

sentences = ["Machine learning is fun.", "It allows computers to learn.", "Summarization helps understand text.", "TF-IDF ranks sentence importance."]
scores = [0.7, 0.9, 0.6, 0.8]

Which two sentences should your summarizer select?

hard

A. ["It allows computers to learn.", "TF-IDF ranks sentence importance."]

B. ["Machine learning is fun.", "Summarization helps understand text."]

C. ["Summarization helps understand text.", "TF-IDF ranks sentence importance."]

D. ["Machine learning is fun.", "It allows computers to learn."]

Solution

1. Step 1: Identify top 2 scores

   The scores are 0.7, 0.9, 0.6, 0.8. The top two are 0.9 and 0.8.

2. Step 2: Match scores to sentences

   0.9 corresponds to "It allows computers to learn.", 0.8 corresponds to "TF-IDF ranks sentence importance.".

3. Final Answer:

   ["It allows computers to learn.", "TF-IDF ranks sentence importance."] -> Option A

4. Quick Check:

   Top 2 scores = 0.9 and 0.8 sentences [OK]

Hint: Pick sentences with highest TF-IDF scores [OK]

Common Mistakes:

• Choosing sentences with lower scores
• Mixing up sentence-score pairs
• Selecting more or fewer than top 2