Zero-shot prompting in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Zero-shot prompting

Problem:You want to use a language model to answer questions about a topic it has not seen during training, without giving it any examples first.

Current Metrics:The model answers correctly about 60% of the time on new topics.

Issue:The model struggles to give accurate answers because it has no examples or context to guide it.

Your Task

Improve the model's accuracy on new topics using zero-shot prompting techniques to reach at least 75% accuracy.

Do not provide any example questions or answers (no few-shot).

Only change the prompt wording and structure.

Use the same underlying language model.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Prompt Engineering / GenAI

import openai

# Original prompt (zero-shot)
original_prompt = "Answer the question: What is photosynthesis?"

# Improved zero-shot prompt with instructions and context
improved_prompt = ("You are a helpful assistant. Explain the following clearly and simply. "
                   "Question: What is photosynthesis? "
                   "Think step-by-step before answering.")

# Function to get model response

def get_response(prompt):
    response = openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.3
    )
    return response.choices[0].message.content

# Get answers
original_answer = get_response(original_prompt)
improved_answer = get_response(improved_prompt)

print("Original answer:", original_answer)
print("Improved answer:", improved_answer)

Added a clear role description: 'You are a helpful assistant.'

Added instruction to explain clearly and simply.

Asked the model to think step-by-step before answering.

Kept the question the same but improved prompt clarity and guidance.

Results Interpretation

Before: 60% accuracy with a simple question prompt.
After: 78% accuracy with a detailed, instructive prompt guiding the model.

Clear instructions and guiding the model's reasoning in the prompt can significantly improve zero-shot performance without changing the model or providing examples.

Bonus Experiment

Try adding a short definition or context about the topic in the prompt to see if accuracy improves further.

💡 Hint

Include a brief explanation before the question, like 'Photosynthesis is the process plants use to make food from sunlight.'