Model selection (GPT-4, GPT-3.5) in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Choosing a model for complex language tasks

You need to select a model for a task that requires understanding subtle context and generating detailed explanations. Which model is best suited?

A GPT-4 because it handles complex context and nuanced language better

B GPT-3.5 because it is faster and cheaper for all tasks

C GPT-3.5 because it has more training data

D GPT-4 because it is optimized for short answers only

💡 Hint

Think about which model is designed for deeper understanding and more detailed outputs.

GPT-4 is designed to better understand complex context and generate nuanced, detailed responses compared to GPT-3.5.

❓ Predict Output

intermediate

Output difference between GPT-3.5 and GPT-4

Given the prompt: 'Explain why the sky is blue in simple terms.' Which model output is more likely to be detailed and accurate?

A GPT-4 output: 'The sky appears blue because molecules in the air scatter blue light from the sun more than other colors.'

B GPT-3.5 output: 'The sky is blue because of the sun.'

C GPT-3.5 output: 'The sky is blue because it reflects the ocean.'

D GPT-4 output: 'The sky is blue because of the moonlight.'

💡 Hint

Look for the option that explains the scientific reason clearly and correctly.

GPT-4 tends to provide more accurate and detailed explanations, while GPT-3.5 may give simpler or less accurate answers.

❓ Hyperparameter

advanced

Choosing model size for cost and performance

You want to balance cost and quality for a chatbot. Which choice best fits a use case needing moderate quality and low cost?

A Use GPT-4 for all queries regardless of cost

B Use GPT-3.5 for simple queries and GPT-4 for complex queries

C Use GPT-4 only for short queries and GPT-3.5 for long queries

D Use GPT-3.5 for all queries to minimize cost

💡 Hint

Consider mixing models based on query complexity to optimize cost and quality.

Using GPT-3.5 for simple queries saves cost, while GPT-4 handles complex queries for better quality.

❓ Metrics

advanced

Evaluating model performance on accuracy

You test GPT-3.5 and GPT-4 on a dataset. GPT-3.5 achieves 75% accuracy, GPT-4 achieves 85%. What does this tell you?

A Accuracy is not useful; only speed matters

B GPT-3.5 is better because lower accuracy means less overfitting

C Both models are equally good because accuracy difference is small

D GPT-4 is better because higher accuracy means it predicts more correctly

💡 Hint

Higher accuracy means the model makes more correct predictions.

Accuracy measures correct predictions; higher accuracy means better performance on the tested data.

🔧 Debug

expert

Identifying cause of unexpected GPT-4 output

You use GPT-4 for a task but get very short, vague answers instead of detailed ones. What is the most likely cause?

A GPT-4 is not capable of detailed answers

B The model is overloaded and returns errors silently

C The prompt is too vague or lacks detail, confusing the model

D You accidentally used GPT-3.5 instead of GPT-4

💡 Hint

Think about how input affects output quality.

GPT-4 needs clear, detailed prompts to generate detailed answers; vague prompts lead to vague outputs.