Why embeddings capture semantic meaning in Prompt Engineering / GenAI - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why do embeddings place similar words close together?

Embeddings are vectors that represent words or items. Why do embeddings place words with similar meanings close to each other in the vector space?

A Because embeddings are randomly assigned and happen to cluster similar words by chance.

B Because embeddings store the exact dictionary definitions of words as numbers.

C Because embeddings are trained to minimize distance between words that appear in similar contexts, capturing their meaning.

D Because embeddings only consider word length and frequency, not meaning.

💡 Hint

Think about how words used in similar sentences might relate.

Embeddings are learned by training models to predict words from their context or vice versa. This causes words used in similar contexts to have vectors close together, capturing semantic similarity.

❓ Predict Output

intermediate

Output of cosine similarity between embeddings

Given two word embeddings as vectors, what is the output of the cosine similarity calculation?

Prompt Engineering / GenAI

import numpy as np

def cosine_similarity(vec1, vec2):
    dot_product = np.dot(vec1, vec2)
    norm1 = np.linalg.norm(vec1)
    norm2 = np.linalg.norm(vec2)
    return dot_product / (norm1 * norm2)

embedding_a = np.array([1, 2, 3])
embedding_b = np.array([2, 4, 6])
result = cosine_similarity(embedding_a, embedding_b)
print(round(result, 2))

A 0.00

B 1.00

C 0.77

D 0.50

💡 Hint

Consider the angle between vectors that are multiples of each other.

Cosine similarity measures the cosine of the angle between two vectors. If one vector is a multiple of the other, the angle is 0 degrees and cosine similarity is 1.

❓ Model Choice

advanced

Which model type best learns semantic embeddings?

Which type of model is best suited to learn embeddings that capture semantic meaning of words?

A A model that predicts the next word given previous words (e.g., language model).

B A model that randomly assigns vectors to words without training.

C A model that only counts word frequency in documents.

D A model that sorts words alphabetically.

💡 Hint

Think about models that learn from context and sequence.

Language models learn to predict words based on context, which helps them learn embeddings that capture semantic relationships.

❓ Hyperparameter

advanced

Effect of embedding dimension size on semantic capture

How does increasing the size of the embedding dimension affect the model's ability to capture semantic meaning?

A Larger dimensions can capture more semantic details but may require more data to train well.

B Larger dimensions always reduce semantic capture due to overfitting.

C Embedding size does not affect semantic meaning capture at all.

D Smaller dimensions always capture more semantic meaning because they are simpler.

💡 Hint

Think about the trade-off between detail and data needed.

Larger embedding sizes allow the model to represent more complex semantic relationships but need more data to avoid overfitting.

❓ Metrics

expert

Choosing the best metric to evaluate semantic similarity of embeddings

Which metric is most appropriate to evaluate how well embeddings capture semantic similarity between words?

A Mean Squared Error (MSE) between embedding vectors.

B Accuracy of predicting the exact word from embedding.

C Counting the number of non-zero elements in embeddings.

D Cosine similarity between embedding vectors.

💡 Hint

Consider metrics that measure angle or direction rather than magnitude.

Cosine similarity measures the angle between vectors, which reflects semantic similarity regardless of vector length.