Embedding generation in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of embedding generation in machine learning?

Embedding generation transforms raw data into a format that machine learning models can understand better. What is the main goal of this process?

A To increase the size of the dataset by duplicating samples

B To convert data into fixed-size vectors capturing semantic meaning

C To remove all noise from the data by filtering

D To convert numerical data into categorical labels

💡 Hint

Think about how words or images are represented so models can work with them.

Embedding generation creates fixed-size vectors that capture the meaning or features of data, making it easier for models to learn patterns.

❓ Predict Output

intermediate

What is the output shape of the embedding vector?

Given the following code snippet generating embeddings for 3 sentences using a model that outputs 768-dimensional vectors, what is the shape of the resulting embedding array?

Prompt Engineering / GenAI

sentences = ['Hello world', 'Machine learning is fun', 'AI helps humans']
embeddings = model.encode(sentences)
print(embeddings.shape)

A (768,)

B (768, 3)

C (3, 3)

D (3, 768)

💡 Hint

Each sentence gets its own vector of length 768.

The model encodes each of the 3 sentences into a 768-dimensional vector, so the output shape is (3, 768).

❓ Model Choice

advanced

Which model type is best suited for generating contextual word embeddings?

You want to generate embeddings that capture the meaning of words depending on their sentence context. Which model type should you choose?

A Transformer-based model like BERT

B Recurrent Neural Network (RNN) without attention

C Simple Bag-of-Words model

D K-Nearest Neighbors (KNN) classifier

💡 Hint

Think about models that understand word order and context deeply.

Transformer-based models like BERT generate embeddings that consider the context of each word in a sentence, unlike simpler models.

❓ Metrics

advanced

Which metric is most appropriate to evaluate similarity between two embedding vectors?

You have two embedding vectors representing sentences. Which metric best measures how similar their meanings are?

A Mean squared error

B Euclidean distance

C Cosine similarity

D Accuracy

💡 Hint

Consider a metric that measures the angle between vectors rather than their length.

Cosine similarity measures the angle between two vectors, effectively capturing similarity in direction regardless of magnitude, which is ideal for embeddings.

🔧 Debug

expert

Why does this embedding generation code raise a TypeError?

Examine the code below that attempts to generate embeddings for a list of texts. Why does it raise a TypeError?

Prompt Engineering / GenAI

texts = ['data science', 'deep learning']
embeddings = model.encode(texts[0], texts[1])

A The encode method expects a single list argument, not multiple string arguments

B The model variable is not defined

C The encode method requires integer inputs, not strings

D The texts list is empty

💡 Hint

Check how the encode method is called and what arguments it expects.

The encode method expects one argument: a list of texts. Passing multiple string arguments causes a TypeError.