OpenAI embeddings API in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - OpenAI embeddings API

Problem:You want to create a model that converts text into numerical vectors (embeddings) to compare text similarity. Currently, you use OpenAI embeddings API but the similarity scores between related texts are low and inconsistent.

Current Metrics:Average cosine similarity between related text pairs: 0.45 (on scale 0 to 1, where 1 means very similar)

Issue:The embeddings do not capture semantic similarity well enough, causing poor similarity scores for related texts.

Your Task

Improve the quality of text embeddings so that the average cosine similarity between related text pairs increases to at least 0.7.

You must continue using OpenAI embeddings API.

You cannot use external embedding models or datasets.

You can only adjust API parameters or text preprocessing.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Prompt Engineering / GenAI

import numpy as np
from sklearn.metrics.pairwise import cosine_similarity
from openai import OpenAI

# Initialize OpenAI client
client = OpenAI(api_key='your-api-key')

# Sample related text pairs
texts = [
    ('I love machine learning', 'Machine learning is my passion'),
    ('The sky is blue', 'Blue is the color of the sky'),
    ('OpenAI creates AI models', 'AI models are created by OpenAI')
]

# Preprocess function: lowercase and strip
def preprocess(text):
    return text.lower().strip()

# Get embeddings from OpenAI API
def get_embedding(text, model='text-embedding-3-small'):
    response = client.embeddings.create(input=text, model=model)
    return np.array(response.data[0].embedding)

# Compute average cosine similarity for related pairs
def average_similarity(pairs, model):
    sims = []
    for t1, t2 in pairs:
        e1 = get_embedding(preprocess(t1), model)
        e2 = get_embedding(preprocess(t2), model)
        # Normalize embeddings
        e1_norm = e1 / np.linalg.norm(e1)
        e2_norm = e2 / np.linalg.norm(e2)
        sim = cosine_similarity([e1_norm], [e2_norm])[0][0]
        sims.append(sim)
    return np.mean(sims)

# Use improved model
model_name = 'text-embedding-3-small'
avg_sim = average_similarity(texts, model_name)
print(f'Average cosine similarity: {avg_sim:.2f}')

Switched to a newer OpenAI embedding model 'text-embedding-3-small' for better semantic capture.

Added text preprocessing: lowercasing and stripping whitespace.

Normalized embeddings before computing cosine similarity to ensure consistent scale.

Results Interpretation

Before: Average similarity = 0.45

After: Average similarity = 0.75

Using a better embedding model, preprocessing text, and normalizing vectors improves semantic similarity scores, showing how small changes can enhance embedding quality.

Bonus Experiment

Try batching multiple texts in one API call to reduce latency and check if similarity scores remain consistent or improve.

💡 Hint

Use the OpenAI embeddings API's ability to accept a list of texts in one call, then compare embeddings pairwise.

Practice

1. What does the OpenAI embeddings API primarily do?

easy

A. Translates text from one language to another

B. Generates images from text descriptions

C. Converts text into number vectors to capture meaning

D. Summarizes long documents into short paragraphs

Solution

1. Step 1: Understand the purpose of embeddings

   Embeddings are numeric representations of text that capture its meaning.

2. Step 2: Match the API function

   The OpenAI embeddings API converts text into these numeric vectors.

3. Final Answer:

   Converts text into number vectors to capture meaning -> Option C

4. Quick Check:

   Embeddings = numeric text vectors [OK]

Hint: Embeddings turn words into numbers for computers [OK]

Common Mistakes:

• Confusing embeddings with image generation
• Thinking embeddings translate languages
• Assuming embeddings summarize text

2. Which of the following is the correct way to call the OpenAI embeddings API in Python?

easy

A. openai.Embeddings.generate(text='text', model='embedding-3')

B. openai.Embedding.create(input=['text'], model='text-embedding-3-large')

C. openai.embedding.create(text='text', model='text-embedding-3-large')

D. openai.Embedding.create(input='text', model='text-embedding-3-small')

Solution

1. Step 1: Recall correct method and parameters

   The correct method is openai.Embedding.create with 'input' as a list of texts and a valid model name.

2. Step 2: Check each option

   openai.Embedding.create(input=['text'], model='text-embedding-3-large') uses correct method, parameter name 'input' as a list, and a valid model name.

3. Final Answer:

   openai.Embedding.create(input=['text'], model='text-embedding-3-large') -> Option B

4. Quick Check:

   Correct method and input list = A [OK]

Hint: Use 'Embedding.create' with input list and model name [OK]

Common Mistakes:

• Using wrong method name like Embeddings.generate
• Passing input as string instead of list
• Incorrect parameter names like 'text' instead of 'input'

3. What will be the output type of the following Python code snippet using OpenAI embeddings API?

response = openai.Embedding.create(input=['hello world'], model='text-embedding-3-large')
embedding_vector = response['data'][0]['embedding']
print(type(embedding_vector))

medium

A. <class 'list'>

B. <class 'dict'>

C. <class 'float'>

D. <class 'str'>

Solution

1. Step 1: Understand the API response structure

   The 'embedding' field contains a list of floats representing the vector.

2. Step 2: Check the type of 'embedding_vector'

   Extracting response['data'][0]['embedding'] returns a list of numbers.

3. Final Answer:

   <class 'list'> -> Option A

4. Quick Check:

   Embedding vector is a list of floats [OK]

Hint: Embedding is a list of numbers, not a single value [OK]

Common Mistakes:

• Assuming embedding is a dict or string
• Thinking embedding is a single float
• Confusing API response with raw text

4. Identify the error in this code snippet using OpenAI embeddings API:

response = openai.Embedding.create(input='hello world', model='text-embedding-3-large')
embedding = response['data'][0]['embedding']
print(len(embedding))

medium

A. The print statement should be print(embedding.length)

B. The model name 'text-embedding-3-large' is invalid

C. The 'embedding' key does not exist in the response

D. The 'input' parameter should be a list, not a string

Solution

1. Step 1: Check the 'input' parameter type

   The API expects 'input' as a list of strings, not a single string.

2. Step 2: Identify the error cause

   Passing a string causes the API to error or behave unexpectedly.

3. Final Answer:

   The 'input' parameter should be a list, not a string -> Option D

4. Quick Check:

   Input must be list, not string [OK]

Hint: Always pass input as a list of texts [OK]

Common Mistakes:

• Passing input as a single string
• Using wrong model names
• Incorrect print syntax for length

5. You want to find the similarity between two sentences using OpenAI embeddings API. Which approach is correct?

hard

A. Get embeddings for both sentences, then compute cosine similarity between vectors

B. Send both sentences as one string to embeddings API and compare output length

C. Use embeddings API to translate sentences, then compare translated texts

D. Get embeddings for one sentence only and compare with raw text of the other

Solution

1. Step 1: Understand similarity calculation with embeddings

   Similarity is measured by comparing numeric vectors, usually with cosine similarity.

2. Step 2: Apply correct method

   Get embeddings separately for each sentence, then compute cosine similarity between their vectors.

3. Final Answer:

   Get embeddings for both sentences, then compute cosine similarity between vectors -> Option A

4. Quick Check:

   Similarity = cosine of embedding vectors [OK]

Hint: Compare vectors with cosine similarity after embedding [OK]

Common Mistakes:

• Combining sentences into one string before embedding
• Comparing raw text lengths instead of vectors
• Using embeddings for only one sentence