System prompts and role setting in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - System prompts and role setting

Problem:You want to create a chatbot that responds clearly and politely by setting a system prompt that defines its role and behavior.

Current Metrics:The chatbot responds inconsistently, sometimes too brief or off-topic, with user satisfaction around 60%.

Issue:The system prompt is missing or too vague, causing the chatbot to not follow a clear role or tone.

Your Task

Improve chatbot responses by designing a clear system prompt that sets its role as a friendly, helpful assistant and maintains polite, clear answers.

You can only change the system prompt text.

You cannot change the chatbot's underlying model or training data.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

system_prompt = "You are a friendly and helpful assistant. Always respond politely and clearly. Keep answers simple and easy to understand. Avoid jargon. If you don't know the answer, say so kindly."

# Example usage in a chatbot framework
chatbot = Chatbot(model='genai-model', system_prompt=system_prompt)

user_input = "Can you explain what machine learning is?"
response = chatbot.generate_response(user_input)
print(response)

Added a clear system prompt defining the chatbot's role as friendly and helpful.

Specified tone to be polite and clear.

Included instructions to avoid jargon and admit when unsure.

Results Interpretation

Before: Responses were inconsistent and sometimes unclear, user satisfaction ~60%.

After: Responses are polite, clear, and consistent, user satisfaction ~85%.

Setting a clear system prompt that defines the AI's role and tone guides it to produce better, more consistent responses.

Bonus Experiment

Try adding example dialogues in the system prompt to further improve response quality.

💡 Hint

Include short example questions and ideal answers in the system prompt to guide the chatbot's style.

Practice

1. What is the main purpose of a system prompt in AI?

easy

A. To tell the AI what role to play

B. To train the AI with new data

C. To fix errors in AI code

D. To speed up AI computations

Solution

1. Step 1: Understand system prompt role

   System prompts guide AI on how to behave or respond.

2. Step 2: Differentiate from other AI tasks

   Training data and code fixes are separate from role setting.

3. Final Answer:

   To tell the AI what role to play -> Option A

4. Quick Check:

   System prompt = role setting [OK]

Hint: System prompts set AI's role or behavior [OK]

Common Mistakes:

• Confusing system prompts with training data
• Thinking system prompts fix AI bugs
• Assuming system prompts speed up AI

2. Which of the following is the correct way to set a system prompt for an AI to act as a tutor?

easy

A. Set prompt = 'You are a helpful assistant.'

B. Set prompt = 'Fix errors in code.'

C. Set prompt = 'Run training on tutor data.'

D. Set prompt = 'You are a tutor who explains simply.'

Solution

1. Step 1: Identify correct prompt style

   The prompt should clearly tell AI to act as a tutor and explain simply.

2. Step 2: Eliminate unrelated options

   Options about training or fixing code are not system prompts.

3. Final Answer:

   Set prompt = 'You are a tutor who explains simply.' -> Option D

4. Quick Check:

   Clear role description = correct prompt [OK]

Hint: Use clear role description in prompt [OK]

Common Mistakes:

• Using vague prompts like 'helpful assistant'
• Confusing prompts with training commands
• Writing prompts unrelated to role

3. Given this system prompt: 'You are a translator from English to Spanish.' What will the AI most likely do when asked 'Hello, how are you?'?

medium

A. Translate it to Spanish

B. Ignore the prompt and answer in English

C. Explain the meaning of the sentence

D. Ask for more context

Solution

1. Step 1: Analyze the system prompt

   The prompt sets AI's role as a translator from English to Spanish.

2. Step 2: Predict AI response to input

   AI will translate the input sentence into Spanish as instructed.

3. Final Answer:

   Translate it to Spanish -> Option A

4. Quick Check:

   Translator prompt = translate output [OK]

Hint: Match prompt role to AI output [OK]

Common Mistakes:

• Thinking AI explains instead of translates
• Assuming AI ignores system prompt
• Expecting AI to ask questions

4. You wrote this system prompt: 'You are a helpful assistant.' but the AI keeps giving very short answers. What is the best fix?

medium

A. Restart the AI server.

B. Remove the system prompt entirely.

C. Change prompt to 'You are a helpful assistant who explains in detail.'

D. Add more training data.

Solution

1. Step 1: Identify problem with prompt

   The prompt is too vague, so AI gives short answers.

2. Step 2: Improve prompt specificity

   Adding 'explains in detail' guides AI to give longer answers.

3. Final Answer:

   Change prompt to 'You are a helpful assistant who explains in detail.' -> Option C

4. Quick Check:

   Specific prompt = better answers [OK]

Hint: Make prompts more specific for better answers [OK]

Common Mistakes:

• Removing prompt instead of improving it
• Thinking training data fixes prompt issues
• Restarting server won't change AI behavior

5. You want the AI to act as a math tutor who only answers questions about addition and subtraction. Which system prompt is best?

hard

A. You are a math tutor who answers all math questions.

B. You are a math tutor who only answers addition and subtraction questions.

C. You are a general assistant.

D. You are a math tutor who answers multiplication questions.

Solution

1. Step 1: Understand the role restriction

   The AI should only answer addition and subtraction questions.

2. Step 2: Choose prompt that limits scope correctly

   You are a math tutor who only answers addition and subtraction questions. clearly restricts AI to addition and subtraction only.

3. Step 3: Eliminate broader or unrelated prompts

   The other options do not restrict to addition and subtraction.

4. Final Answer:

   You are a math tutor who only answers addition and subtraction questions. -> Option B

5. Quick Check:

   Specific role limits AI scope [OK]

Hint: Use clear limits in prompt for focused AI roles [OK]

Common Mistakes:

• Using broad prompts without limits
• Choosing unrelated math topics
• Not specifying question types