Model selection (GPT-4, GPT-3.5) in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Choosing a model for complex language tasks

You need to select a model for a task that requires understanding subtle context and generating detailed explanations. Which model is best suited?

A GPT-4 because it handles complex context and nuanced language better

B GPT-3.5 because it is faster and cheaper for all tasks

C GPT-3.5 because it has more training data

D GPT-4 because it is optimized for short answers only

💡 Hint

Think about which model is designed for deeper understanding and more detailed outputs.

GPT-4 is designed to better understand complex context and generate nuanced, detailed responses compared to GPT-3.5.

❓ Predict Output

intermediate

Output difference between GPT-3.5 and GPT-4

Given the prompt: 'Explain why the sky is blue in simple terms.' Which model output is more likely to be detailed and accurate?

A GPT-4 output: 'The sky appears blue because molecules in the air scatter blue light from the sun more than other colors.'

B GPT-3.5 output: 'The sky is blue because of the sun.'

C GPT-3.5 output: 'The sky is blue because it reflects the ocean.'

D GPT-4 output: 'The sky is blue because of the moonlight.'

💡 Hint

Look for the option that explains the scientific reason clearly and correctly.

GPT-4 tends to provide more accurate and detailed explanations, while GPT-3.5 may give simpler or less accurate answers.

❓ Hyperparameter

advanced

Choosing model size for cost and performance

You want to balance cost and quality for a chatbot. Which choice best fits a use case needing moderate quality and low cost?

A Use GPT-4 for all queries regardless of cost

B Use GPT-3.5 for simple queries and GPT-4 for complex queries

C Use GPT-4 only for short queries and GPT-3.5 for long queries

D Use GPT-3.5 for all queries to minimize cost

💡 Hint

Consider mixing models based on query complexity to optimize cost and quality.

Using GPT-3.5 for simple queries saves cost, while GPT-4 handles complex queries for better quality.

❓ Metrics

advanced

Evaluating model performance on accuracy

You test GPT-3.5 and GPT-4 on a dataset. GPT-3.5 achieves 75% accuracy, GPT-4 achieves 85%. What does this tell you?

A Accuracy is not useful; only speed matters

B GPT-3.5 is better because lower accuracy means less overfitting

C Both models are equally good because accuracy difference is small

D GPT-4 is better because higher accuracy means it predicts more correctly

💡 Hint

Higher accuracy means the model makes more correct predictions.

Accuracy measures correct predictions; higher accuracy means better performance on the tested data.

🔧 Debug

expert

Identifying cause of unexpected GPT-4 output

You use GPT-4 for a task but get very short, vague answers instead of detailed ones. What is the most likely cause?

A GPT-4 is not capable of detailed answers

B The model is overloaded and returns errors silently

C The prompt is too vague or lacks detail, confusing the model

D You accidentally used GPT-3.5 instead of GPT-4

💡 Hint

Think about how input affects output quality.

GPT-4 needs clear, detailed prompts to generate detailed answers; vague prompts lead to vague outputs.

Practice

1. Which model should you choose if you need detailed and complex text generation?

easy

A. GPT-3.5

B. Both are equally detailed

C. GPT-4

D. Neither, use a smaller model

Solution

1. Step 1: Understand model capabilities

   GPT-4 is designed for more complex and detailed tasks compared to GPT-3.5.

2. Step 2: Match task complexity to model

   For detailed and complex text generation, GPT-4 is the better choice.

3. Final Answer:

   GPT-4 -> Option C

4. Quick Check:

   Complex tasks = GPT-4 [OK]

Hint: Choose GPT-4 for complexity, GPT-3.5 for speed [OK]

Common Mistakes:

• Choosing GPT-3.5 for complex tasks
• Thinking both models have same detail level

2. Which of the following is the correct way to specify GPT-3.5 in an API call?

easy

A. "model": "gpt-3.5-turbo"

B. "model": "gpt-3"

C. "model": "gpt-4"

D. "model": "gpt-5"

Solution

1. Step 1: Recall model naming conventions

   The GPT-3.5 model is named "gpt-3.5-turbo" in API calls.

2. Step 2: Identify correct option

   "model": "gpt-3.5-turbo" matches the exact model name for GPT-3.5.

3. Final Answer:

   "model": "gpt-3.5-turbo" -> Option A

4. Quick Check:

   Correct model name = "model": "gpt-3.5-turbo" [OK]

Hint: Use exact model name string in API call [OK]

Common Mistakes:

• Using "gpt-3" instead of "gpt-3.5-turbo"
• Confusing GPT-4 name with GPT-3.5

3. Given this code snippet calling the OpenAI API, which model will produce faster responses but possibly less detailed output?

response = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[{"role": "user", "content": "Explain photosynthesis."}]
)

medium

A. GPT-3.5, faster but less detailed

B. GPT-4, slower but more detailed

C. GPT-4, faster and more detailed

D. GPT-3.5, slower but more detailed

Solution

1. Step 1: Identify the model used in code

   The code uses "gpt-3.5-turbo" as the model parameter.

2. Step 2: Recall model speed and detail tradeoff

   GPT-3.5 is faster but less detailed compared to GPT-4.

3. Final Answer:

   GPT-3.5, faster but less detailed -> Option A

4. Quick Check:

   Model in code = GPT-3.5 [OK]

Hint: Check model name string to know speed/detail tradeoff [OK]

Common Mistakes:

• Assuming GPT-3.5 is slower
• Confusing model names in code snippet

4. You wrote this API call but get an error:

response = openai.ChatCompletion.create(
  model="gpt-3.5",
  messages=[{"role": "user", "content": "Tell me a joke."}]
)

What is the likely problem?

medium

A. Messages list is missing a system role

B. Model name is incomplete, should be "gpt-3.5-turbo"

C. API key is missing

D. The model "gpt-3.5" does not exist

Solution

1. Step 1: Check model name correctness

   The model name "gpt-3.5" is incomplete; the correct full name is "gpt-3.5-turbo".

2. Step 2: Understand error cause

   Using an incomplete model name causes the API to reject the call.

3. Final Answer:

   Model name is incomplete, should be "gpt-3.5-turbo" -> Option B

4. Quick Check:

   Model name must be exact [OK]

Hint: Use full model name string to avoid errors [OK]

Common Mistakes:

• Using partial model names
• Assuming system role is mandatory
• Ignoring API key errors

5. You want to build a chatbot that answers customer questions quickly and cheaply but can switch to detailed answers when needed. How should you select models in your code?

hard

A. Always use GPT-4 for all answers

B. Use GPT-4 only, it is always more accurate

C. Use GPT-3.5 only, it is always faster and cheaper

D. Use GPT-3.5 for quick replies, switch to GPT-4 for detailed ones

Solution

1. Step 1: Understand tradeoffs between GPT-3.5 and GPT-4

   GPT-3.5 is faster and cheaper but less detailed; GPT-4 is slower and costlier but more detailed.

2. Step 2: Match chatbot needs to model selection

   Use GPT-3.5 for quick, cheap answers and switch to GPT-4 when detailed responses are needed.

3. Final Answer:

   Use GPT-3.5 for quick replies, switch to GPT-4 for detailed ones -> Option D

4. Quick Check:

   Balance speed and detail with model switching [OK]

Hint: Switch models based on answer detail needed [OK]

Common Mistakes:

• Using only one model for all tasks
• Ignoring cost and speed tradeoffs