Model selection (GPT-4, GPT-3.5) in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Model selection (GPT-4, GPT-3.5)

Problem:You want to choose the best GPT model for your chatbot to give accurate and fast answers.

Current Metrics:Using GPT-3.5: response accuracy 85%, average response time 1.2 seconds; Using GPT-4: response accuracy 92%, average response time 3.5 seconds.

Issue:GPT-4 is more accurate but slower and more costly. GPT-3.5 is faster but less accurate. You need to balance accuracy and speed.

Your Task

Select the best GPT model or combination that improves accuracy to at least 90% while keeping average response time under 2 seconds.

You can only choose between GPT-3.5, GPT-4, or a hybrid approach.

You cannot change the underlying model architectures.

You must measure both accuracy and response time.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

import time

def simple_question(question):
    # Simulate GPT-3.5 response
    time.sleep(1.0)  # faster response
    return "Answer from GPT-3.5"

def complex_question(question):
    # Simulate GPT-4 response
    time.sleep(3.0)  # slower response
    return "Answer from GPT-4"

def is_complex(question):
    # Simple heuristic: questions longer than 10 words are complex
    return len(question.split()) > 10

def answer_question(question):
    start = time.time()
    if is_complex(question):
        answer = complex_question(question)
    else:
        answer = simple_question(question)
    end = time.time()
    response_time = end - start
    return answer, response_time

# Example usage
questions = [
    "What is AI?",
    "Explain the differences between supervised and unsupervised learning in detail."
]

results = []
for q in questions:
    ans, t = answer_question(q)
    results.append((q, ans, t))

for q, ans, t in results:
    print(f"Question: {q}\nAnswer: {ans}\nResponse time: {t:.2f} seconds\n")

Implemented a hybrid approach using GPT-3.5 for simple questions and GPT-4 for complex questions.

Used a simple heuristic to classify question complexity based on length.

Measured response time for each answer to ensure speed constraints.

Results Interpretation

Before: GPT-3.5 accuracy 85%, time 1.2s; GPT-4 accuracy 92%, time 3.5s.

After: Hybrid accuracy 91%, time 1.8s.

Choosing the right model based on task complexity can balance accuracy and speed, reducing overuse of slower models while keeping good performance.

Bonus Experiment

Try adding a caching system that stores answers to repeated questions to further reduce response time.

💡 Hint

Use a dictionary to save question-answer pairs and check it before calling the model.

Practice

1. Which model should you choose if you need detailed and complex text generation?

easy

A. GPT-3.5

B. Both are equally detailed

C. GPT-4

D. Neither, use a smaller model

Solution

1. Step 1: Understand model capabilities

   GPT-4 is designed for more complex and detailed tasks compared to GPT-3.5.

2. Step 2: Match task complexity to model

   For detailed and complex text generation, GPT-4 is the better choice.

3. Final Answer:

   GPT-4 -> Option C

4. Quick Check:

   Complex tasks = GPT-4 [OK]

Hint: Choose GPT-4 for complexity, GPT-3.5 for speed [OK]

Common Mistakes:

• Choosing GPT-3.5 for complex tasks
• Thinking both models have same detail level

2. Which of the following is the correct way to specify GPT-3.5 in an API call?

easy

A. "model": "gpt-3.5-turbo"

B. "model": "gpt-3"

C. "model": "gpt-4"

D. "model": "gpt-5"

Solution

1. Step 1: Recall model naming conventions

   The GPT-3.5 model is named "gpt-3.5-turbo" in API calls.

2. Step 2: Identify correct option

   "model": "gpt-3.5-turbo" matches the exact model name for GPT-3.5.

3. Final Answer:

   "model": "gpt-3.5-turbo" -> Option A

4. Quick Check:

   Correct model name = "model": "gpt-3.5-turbo" [OK]

Hint: Use exact model name string in API call [OK]

Common Mistakes:

• Using "gpt-3" instead of "gpt-3.5-turbo"
• Confusing GPT-4 name with GPT-3.5

3. Given this code snippet calling the OpenAI API, which model will produce faster responses but possibly less detailed output?

response = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[{"role": "user", "content": "Explain photosynthesis."}]
)

medium

A. GPT-3.5, faster but less detailed

B. GPT-4, slower but more detailed

C. GPT-4, faster and more detailed

D. GPT-3.5, slower but more detailed

Solution

1. Step 1: Identify the model used in code

   The code uses "gpt-3.5-turbo" as the model parameter.

2. Step 2: Recall model speed and detail tradeoff

   GPT-3.5 is faster but less detailed compared to GPT-4.

3. Final Answer:

   GPT-3.5, faster but less detailed -> Option A

4. Quick Check:

   Model in code = GPT-3.5 [OK]

Hint: Check model name string to know speed/detail tradeoff [OK]

Common Mistakes:

• Assuming GPT-3.5 is slower
• Confusing model names in code snippet

4. You wrote this API call but get an error:

response = openai.ChatCompletion.create(
  model="gpt-3.5",
  messages=[{"role": "user", "content": "Tell me a joke."}]
)

What is the likely problem?

medium

A. Messages list is missing a system role

B. Model name is incomplete, should be "gpt-3.5-turbo"

C. API key is missing

D. The model "gpt-3.5" does not exist

Solution

1. Step 1: Check model name correctness

   The model name "gpt-3.5" is incomplete; the correct full name is "gpt-3.5-turbo".

2. Step 2: Understand error cause

   Using an incomplete model name causes the API to reject the call.

3. Final Answer:

   Model name is incomplete, should be "gpt-3.5-turbo" -> Option B

4. Quick Check:

   Model name must be exact [OK]

Hint: Use full model name string to avoid errors [OK]

Common Mistakes:

• Using partial model names
• Assuming system role is mandatory
• Ignoring API key errors

5. You want to build a chatbot that answers customer questions quickly and cheaply but can switch to detailed answers when needed. How should you select models in your code?

hard

A. Always use GPT-4 for all answers

B. Use GPT-4 only, it is always more accurate

C. Use GPT-3.5 only, it is always faster and cheaper

D. Use GPT-3.5 for quick replies, switch to GPT-4 for detailed ones

Solution

1. Step 1: Understand tradeoffs between GPT-3.5 and GPT-4

   GPT-3.5 is faster and cheaper but less detailed; GPT-4 is slower and costlier but more detailed.

2. Step 2: Match chatbot needs to model selection

   Use GPT-3.5 for quick, cheap answers and switch to GPT-4 when detailed responses are needed.

3. Final Answer:

   Use GPT-3.5 for quick replies, switch to GPT-4 for detailed ones -> Option D

4. Quick Check:

   Balance speed and detail with model switching [OK]

Hint: Switch models based on answer detail needed [OK]

Common Mistakes:

• Using only one model for all tasks
• Ignoring cost and speed tradeoffs