Iterative prompt refinement in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Iterative prompt refinement

Problem:You want to get better answers from a text generation AI by improving your prompts step-by-step.

Current Metrics:Initial prompt yields 60% relevance and 50% clarity in AI responses.

Issue:The AI responses are not very relevant or clear because the prompt is too vague and not specific enough.

Your Task

Improve the prompt iteratively to increase response relevance and clarity to at least 85%.

You can only change the prompt text, not the AI model or settings.

You must keep the prompt length under 100 words.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

import openai

# Initial prompt
prompt = "Tell me about dogs."

# Function to get AI response

def get_response(prompt_text):
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt_text}]
    )
    return response.choices[0].message.content

# Iteration 1: Add specificity
prompt1 = "Tell me about the common breeds of dogs and their characteristics."
response1 = get_response(prompt1)

# Iteration 2: Ask for examples and explanation
prompt2 = "Tell me about the common breeds of dogs, their characteristics, and give examples of their behavior."
response2 = get_response(prompt2)

# Iteration 3: Request clarity and summary
prompt3 = "Explain the common dog breeds, their characteristics, examples of behavior, and provide a short summary at the end."
response3 = get_response(prompt3)

print("Iteration 1 Response:", response1)
print("Iteration 2 Response:", response2)
print("Iteration 3 Response:", response3)

Added specific details about dog breeds and characteristics to the prompt.

Included request for examples of behavior to improve relevance.

Asked for explanation and a summary to enhance clarity.

Results Interpretation

Before: Relevance 60%, Clarity 50%
After: Relevance 88%, Clarity 90%

Improving AI responses can be done by refining prompts step-by-step, adding details, examples, and clear instructions.

Bonus Experiment

Try refining prompts for a different topic, like cooking recipes, to improve AI response quality.

💡 Hint

Use specific ingredients, cooking methods, and ask for step-by-step instructions.

Practice

1. What is the main goal of iterative prompt refinement when working with AI models?

easy

A. To avoid changing the prompt once it is written

B. To improve the prompt step-by-step for clearer AI answers

C. To write the longest possible prompt in one try

D. To use random words to confuse the AI

Solution

1. Step 1: Understand the purpose of prompt refinement

   Iterative prompt refinement means making small changes to your prompt to get better AI responses.

2. Step 2: Identify the goal of this process

   The goal is to improve clarity and usefulness of AI answers by adjusting the prompt step-by-step.

3. Final Answer:

   To improve the prompt step-by-step for clearer AI answers -> Option B

4. Quick Check:

   Iterative refinement = step-by-step improvement [OK]

Hint: Think: How do you get better answers? By improving prompts stepwise [OK]

Common Mistakes:

• Thinking the prompt should never change
• Believing longer prompts always work best
• Assuming random words help AI understand

2. Which of the following is the correct way to start refining a prompt iteratively?

easy

A. Write a clear initial prompt and test AI response

B. Write a very long prompt with many unrelated details

C. Use only one word as a prompt

D. Never test the prompt before finalizing

Solution

1. Step 1: Identify best practice for starting prompt refinement

   Start with a clear, simple prompt to see how AI responds.

2. Step 2: Understand why testing matters

   Testing helps know what to improve next in the prompt.

3. Final Answer:

   Write a clear initial prompt and test AI response -> Option A

4. Quick Check:

   Start clear + test = best first step [OK]

Hint: Begin with clarity and test before changing [OK]

Common Mistakes:

• Starting with confusing or too long prompts
• Skipping testing before refining
• Using too few words to explain

3. Given this initial prompt: "List fruits", after iterative refinement, which prompt is likely to get a better AI answer listing only tropical fruits?

medium

A. "List fruits"

B. "List all fruits and animals"

C. "Tell me about fruits and vegetables"

D. "List tropical fruits only"

Solution

1. Step 1: Compare initial and refined prompts

   The initial prompt "List fruits" is broad and may list all fruits.

2. Step 2: Identify which prompt narrows the request

   "List tropical fruits only" clearly asks for tropical fruits, refining the request.

3. Final Answer:

   "List tropical fruits only" -> Option D

4. Quick Check:

   Specific prompt = better targeted answer [OK]

Hint: Add specific details to focus AI answers [OK]

Common Mistakes:

• Choosing too broad prompts
• Mixing unrelated topics in prompt
• Not specifying the desired subset

4. You wrote this prompt: "Explain AI" but the AI gives a very technical answer. What is the best fix using iterative prompt refinement?

medium

A. Use unrelated words like 'banana' in the prompt

B. Add more technical terms to the prompt

C. Change prompt to 'Explain AI in simple words for beginners'

D. Make the prompt shorter to just 'AI'

Solution

1. Step 1: Identify the problem with the original prompt

   Original prompt is too broad, causing a technical answer that may be hard to understand.

2. Step 2: Choose a refinement that clarifies the audience

   Adding 'in simple words for beginners' guides AI to simplify the explanation.

3. Final Answer:

   Change prompt to 'Explain AI in simple words for beginners' -> Option C

4. Quick Check:

   Clarify audience to simplify AI response [OK]

Hint: Specify audience or style to guide AI tone [OK]

Common Mistakes:

• Adding unrelated words confuses AI
• Making prompt too short loses context
• Adding more technical terms worsens complexity

5. You want the AI to generate a list of 5 healthy breakfast ideas but it keeps giving snacks. Which iterative prompt refinement will best fix this?

hard

A. "List 5 healthy breakfast ideas only, no snacks"

B. "List 5 snacks and breakfast ideas"

C. "List any 5 food items"

D. "List 5 unhealthy breakfast ideas"

Solution

1. Step 1: Identify the issue with the current AI output

   The AI includes snacks because the prompt is not specific enough to exclude them.

2. Step 2: Refine the prompt to exclude snacks and focus on healthy breakfasts

   Adding 'only, no snacks' clearly tells AI to avoid snacks and focus on breakfast ideas.

3. Final Answer:

   "List 5 healthy breakfast ideas only, no snacks" -> Option A

4. Quick Check:

   Clear exclusions improve AI focus [OK]

Hint: Use clear exclusions to avoid unwanted answers [OK]

Common Mistakes:

• Including snacks by not excluding them
• Being too vague about food types
• Requesting unhealthy options by mistake