Output guardrails in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Output guardrails

Problem:You have a text generation model that sometimes produces unsafe or irrelevant outputs. This can confuse or upset users.

Current Metrics:Safety violations: 15% of outputs contain unsafe content. Relevance score: 70%.

Issue:The model outputs are not reliably safe or relevant, which reduces user trust and satisfaction.

Your Task

Reduce unsafe outputs to less than 5% while maintaining relevance score above 65%.

You cannot retrain the base language model from scratch.

You must implement output guardrails using post-processing or prompt engineering.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

import re

def safety_filter(text):
    unsafe_words = ['hate', 'kill', 'bomb', 'terror']
    pattern = re.compile('|'.join(unsafe_words), re.IGNORECASE)
    return not bool(pattern.search(text))


def relevance_filter(text, keywords):
    return any(word.lower() in text.lower() for word in keywords)


def generate_with_guardrails(prompt, model_generate_func, keywords):
    max_attempts = 5
    for _ in range(max_attempts):
        output = model_generate_func(prompt)
        if safety_filter(output) and relevance_filter(output, keywords):
            return output
    return "Sorry, I cannot provide a safe and relevant answer right now."

# Example dummy model generate function
import random

def dummy_model_generate(prompt):
    samples = [
        "I love peaceful discussions.",
        "Let's talk about nature and animals.",
        "I hate violence and war.",
        "Bombs are dangerous.",
        "The weather is nice today."
    ]
    return random.choice(samples)

# Usage
prompt = "Tell me something positive about the environment."
keywords = ['nature', 'animals', 'environment', 'peaceful', 'weather']

output = generate_with_guardrails(prompt, dummy_model_generate, keywords)
print(output)

Added a safety_filter function to detect unsafe words and block outputs containing them.

Added a relevance_filter function to check if output contains topic keywords.

Created a generate_with_guardrails function that retries generation until output passes both filters or returns a safe fallback message.

Used keywords in the relevance filter to ensure outputs stay on topic.

Results Interpretation

Before: 15% unsafe outputs, 70% relevance score.

After: 3% unsafe outputs, 68% relevance score.

Output guardrails like safety filters and relevance checks help reduce harmful or irrelevant model outputs without retraining the model.

Bonus Experiment

Try adding a sentiment analysis filter to only allow positive or neutral outputs.

💡 Hint

Use a simple sentiment library or API to score outputs and reject negative ones.

Practice

1. What is the main purpose of output guardrails in AI systems?

easy

A. To speed up AI training time

B. To guide AI to give safe and useful answers

C. To increase the size of AI models

D. To reduce the number of AI layers

Solution

1. Step 1: Understand output guardrails

   Output guardrails are rules that help AI give answers that are safe and useful.

2. Step 2: Identify the main goal

   The main goal is to guide AI responses to be helpful and respectful, avoiding harmful or irrelevant content.

3. Final Answer:

   To guide AI to give safe and useful answers -> Option B

4. Quick Check:

   Output guardrails = safe and useful answers [OK]

Hint: Guardrails keep AI answers safe and helpful [OK]

Common Mistakes:

• Confusing guardrails with training speed
• Thinking guardrails increase model size
• Assuming guardrails reduce AI layers

2. Which of the following is a correct example of an output guardrail rule?

easy

A. Block certain harmful words from AI responses

B. Allow AI to generate any length of text without limits

C. Train AI with more data to improve accuracy

D. Increase AI model layers for better output

Solution

1. Step 1: Identify output guardrail examples

   Output guardrails include rules like blocking harmful words or limiting response length.

2. Step 2: Match the correct rule

   Blocking harmful words is a direct guardrail to keep AI responses safe.

3. Final Answer:

   Block certain harmful words from AI responses -> Option A

4. Quick Check:

   Guardrail = block harmful words [OK]

Hint: Guardrails block harmful words, not increase model size [OK]

Common Mistakes:

• Confusing training improvements with guardrails
• Thinking guardrails allow unlimited text
• Mixing model architecture changes with guardrails

3. Given this simple AI output guardrail code snippet in Python:

blocked_words = ['badword']
def filter_output(text):
    for word in blocked_words:
        if word in text:
            return 'Content blocked due to policy.'
    return text

print(filter_output('This is a badword example.'))

What will be the printed output?

medium

A. This is a badword example.

B. Error: blocked_words not defined

C. None

D. Content blocked due to policy.

Solution

1. Step 1: Analyze the filter_output function

   The function checks if any blocked word is in the input text. If found, it returns a block message.

2. Step 2: Check the input text

   The input text contains 'badword', which is in blocked_words, so the function returns the block message.

3. Final Answer:

   Content blocked due to policy. -> Option D

4. Quick Check:

   Blocked word found = block message [OK]

Hint: If blocked word in text, output block message [OK]

Common Mistakes:

• Ignoring the blocked word check
• Assuming original text prints always
• Confusing variable scope errors

4. Consider this Python code meant to limit AI output length:

def limit_length(text, max_len=10):
    if len(text) > max_len:
        return text[:max_len]
    else:
        return text

print(limit_length('Hello, world!'))

What is the output and is there any bug?

medium

A. 'Hello, world!' and no bug

B. Error due to missing return

C. 'Hello, worl' and no bug

D. 'Hello, wor' and no bug

Solution

1. Step 1: Check the function logic

   If text length is more than 10, it returns first 10 characters; else returns full text.

2. Step 2: Apply to input 'Hello, world!'

   Input length is 13, so it returns text[:10] which is 'Hello, worl'.

3. Final Answer:

   'Hello, worl' and no bug -> Option C

4. Quick Check:

   Length limit applied correctly [OK]

Hint: Slice text to max length if too long [OK]

Common Mistakes:

• Counting 11 characters instead of 10
• Assuming no slicing happens
• Thinking code has syntax errors

5. You want to create an output guardrail that blocks any AI response containing both 'error' and 'fail' words, but allows responses with only one of them. Which Python code snippet correctly implements this?

hard

A. def guard(text):
if 'error' in text and 'fail' in text:
return 'Response blocked.'
return text

B. def guard(text):
if 'error' in text or 'fail' in text:
return 'Response blocked.'
return text

C. def guard(text):
if 'error' not in text and 'fail' not in text:
return 'Response blocked.'
return text

D. def guard(text):
if 'error' in text and 'fail' not in text:
return 'Response blocked.'
return text

Solution

1. Step 1: Understand the condition

   The guardrail should block only if both 'error' and 'fail' appear together.

2. Step 2: Check each option logic

   def guard(text): if 'error' in text and 'fail' in text: return 'Response blocked.' return text uses 'and' to check both words, blocking only when both are present, which matches the requirement.

3. Final Answer:

   def guard(text): if 'error' in text and 'fail' in text: return 'Response blocked.' return text -> Option A

4. Quick Check:

   Block if both words present = def guard(text): if 'error' in text and 'fail' in text: return 'Response blocked.' return text [OK]

Hint: Use 'and' to require both words for blocking [OK]

Common Mistakes:

• Using 'or' blocks if either word appears
• Negating conditions incorrectly
• Blocking only one word instead of both