Sentence-BERT for embeddings in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to load the Sentence-BERT model for embeddings.

NLP

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('[1]')

Drag options to blanks, or click blank then click option'

Abert-base-uncased

Broberta-base

Cgpt2

Dall-MiniLM-L6-v2

Attempts:

3 left

💡 Hint

Common Mistakes

Using a standard BERT model name instead of a Sentence-BERT model.

Using a language model like GPT-2 which is not for sentence embeddings.

The 'all-MiniLM-L6-v2' model is a popular Sentence-BERT model for generating sentence embeddings.

2fill in blank

medium

Complete the code to generate embeddings for a list of sentences.

NLP

sentences = ['I love machine learning.', 'Sentence embeddings are useful.']
embeddings = model.[1](sentences)

Drag options to blanks, or click blank then click option'

Aencode

Bpredict

Ctransform

Dfit

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'predict' which is for classification models.

Using 'fit' which is for training models.

The 'encode' method generates embeddings from sentences using the Sentence-BERT model.

3fill in blank

hard

Fix the error in the code to correctly compute cosine similarity between two embeddings.

NLP

from sklearn.metrics.pairwise import [1]
similarity = cosine_similarity([embeddings[0]], [embeddings[1]])[0][0]

Drag options to blanks, or click blank then click option'

Acosine_similarity

Beuclidean_distances

Cpairwise_distances

Dmanhattan_distances

Attempts:

3 left

💡 Hint

Common Mistakes

Using distance functions instead of similarity functions.

Using functions that return distances rather than similarity scores.

The 'cosine_similarity' function computes the cosine similarity between vectors, which is commonly used for embeddings.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension mapping sentences to their embedding lengths.

NLP

lengths = {sentence: len(embedding) for sentence, embedding in zip(sentences, [1])}
filtered = {k: v for k, v in lengths.items() if v [2] 384}

Drag options to blanks, or click blank then click option'

Aembeddings

B>

C<

Dsentences

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'sentences' instead of 'embeddings' in the zip.

Using '<' instead of '>' for filtering.

We use 'embeddings' to get the vectors and filter those with length greater than 384.

5fill in blank

hard

Fill all three blanks to create a dictionary of sentences and their cosine similarity scores above 0.7 with the first sentence.

NLP

from sklearn.metrics.pairwise import [1]
scores = {sentence: [2]([embeddings[0]], [embedding])[0][0] for sentence, embedding in zip(sentences, embeddings)}
filtered_scores = {k: v for k, v in scores.items() if v [3] 0.7}

Drag options to blanks, or click blank then click option'

Acosine_similarity

Bcosine_distances

C>

D<

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'cosine_distances' which returns distances, not similarity.

Using '<' instead of '>' for filtering.

We import and use 'cosine_similarity' to compute scores and filter those greater than 0.7.

Practice

1. What is the main purpose of Sentence-BERT embeddings in NLP?

easy

A. To count the number of words in a sentence

B. To translate sentences into different languages

C. To generate random sentences for data augmentation

D. To convert sentences into numbers that capture their meaning

Solution

1. Step 1: Understand Sentence-BERT's role

   Sentence-BERT creates embeddings, which are numbers representing sentence meaning.

2. Step 2: Compare options with Sentence-BERT's function

   Only To convert sentences into numbers that capture their meaning describes converting sentences into meaningful numbers, matching Sentence-BERT's purpose.

3. Final Answer:

   To convert sentences into numbers that capture their meaning -> Option D

4. Quick Check:

   Sentence-BERT embeddings = meaningful numbers [OK]

Hint: Remember: embeddings = numbers capturing meaning [OK]

Common Mistakes:

• Confusing embeddings with translation
• Thinking embeddings count words
• Assuming embeddings generate sentences

2. Which Python code snippet correctly loads a pre-trained Sentence-BERT model using the sentence-transformers library?

easy

A. from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')

B. import sentence_transformers
model = sentence_transformers.load('all-MiniLM-L6-v2')

C. from transformers import SentenceBert
model = SentenceBert.load('all-MiniLM-L6-v2')

D. import sbert
model = sbert.SentenceTransformer('all-MiniLM-L6-v2')

Solution

1. Step 1: Recall correct import and model loading syntax

   The sentence-transformers library uses 'from sentence_transformers import SentenceTransformer' and then creates a model instance with the model name.

2. Step 2: Check each option for correctness

   from sentence_transformers import SentenceTransformer model = SentenceTransformer('all-MiniLM-L6-v2') matches the correct syntax. Options A, B, and D use incorrect imports or methods.

3. Final Answer:

   from sentence_transformers import SentenceTransformer model = SentenceTransformer('all-MiniLM-L6-v2') -> Option A

4. Quick Check:

   Correct import and model load = from sentence_transformers import SentenceTransformer model = SentenceTransformer('all-MiniLM-L6-v2') [OK]

Hint: Use 'from sentence_transformers import SentenceTransformer' [OK]

Common Mistakes:

• Using wrong import statements
• Calling non-existent load methods
• Confusing transformers library with sentence-transformers

3. Given the code below, what is the output shape of embeddings?

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')
sentences = ['Hello world', 'How are you?']
embeddings = model.encode(sentences)
print(embeddings.shape)

medium

A. (384, 2)

B. (2, 384)

C. (2, 768)

D. (1, 384)

Solution

1. Step 1: Understand input and output of model.encode()

   Input is 2 sentences, so output embeddings will have 2 rows, one per sentence.

2. Step 2: Know embedding dimension of 'all-MiniLM-L6-v2'

   This model produces embeddings of size 384 per sentence.

3. Final Answer:

   (2, 384) -> Option B

4. Quick Check:

   2 sentences x 384 dims = (2, 384) [OK]

Hint: Output shape = (number of sentences, embedding size) [OK]

Common Mistakes:

• Swapping dimensions in output shape
• Assuming embedding size is 768
• Forgetting batch size dimension

4. You run this code but get an error: AttributeError: module 'sentence_transformers' has no attribute 'load'. What is the likely cause?

import sentence_transformers
model = sentence_transformers.load('all-MiniLM-L6-v2')

medium

A. The model file is missing from local directory

B. The model name 'all-MiniLM-L6-v2' is incorrect

C. The sentence_transformers module does not have a 'load' function

D. You need to import SentenceTransformer class explicitly

Solution

1. Step 1: Analyze the error message

   The error says 'sentence_transformers' has no attribute 'load', meaning 'load' is not a valid function in this module.

2. Step 2: Understand correct usage

   The correct way is to import SentenceTransformer class and instantiate it with the model name, not use 'load'.

3. Final Answer:

   The sentence_transformers module does not have a 'load' function -> Option C

4. Quick Check:

   AttributeError means wrong function call [OK]

Hint: Use SentenceTransformer(), not load() [OK]

Common Mistakes:

• Calling non-existent 'load' method
• Not importing SentenceTransformer class
• Assuming model loads from local file by default

5. You want to find the most similar sentence to 'I love machine learning' from a list using Sentence-BERT embeddings. Which approach is best?

hard

A. Encode all sentences and query, then find the sentence with highest cosine similarity to the query embedding

B. Count common words between query and each sentence, pick the highest count

C. Use a pre-trained translation model to translate sentences before comparison

D. Encode only the query sentence and compare it to raw text sentences

Solution

1. Step 1: Understand how Sentence-BERT embeddings are used for similarity

   Sentence-BERT embeddings represent sentence meaning as vectors; similarity is measured by cosine similarity between vectors.

2. Step 2: Evaluate options for similarity search

   Encode all sentences and query, then find the sentence with highest cosine similarity to the query embedding correctly encodes all sentences and compares embeddings using cosine similarity. Other options do not use embeddings properly or rely on less effective methods.

3. Final Answer:

   Encode all sentences and query, then find the sentence with highest cosine similarity to the query embedding -> Option A

4. Quick Check:

   Embedding + cosine similarity = best similarity search [OK]

Hint: Compare embeddings with cosine similarity for best match [OK]

Common Mistakes:

• Comparing raw text instead of embeddings
• Using word count instead of semantic similarity
• Encoding only query, not all sentences