Prompt injection defense in Prompt Engineering / GenAI - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is prompt injection in AI language models?

Prompt injection is when someone adds unexpected or harmful instructions into the input prompt to trick the AI into doing something unintended.

beginner

Why is prompt injection a problem for AI systems?

Because it can make the AI give wrong, harmful, or unsafe answers by tricking it with hidden commands inside the prompt.

intermediate

Name one simple way to defend against prompt injection.

One way is to carefully filter or clean the input prompt to remove suspicious or harmful instructions before sending it to the AI.

intermediate

What does 'context isolation' mean in prompt injection defense?

It means keeping the user input separate from the AI's instructions so the user cannot change or add commands that affect the AI's behavior.

intermediate

How can monitoring AI outputs help defend against prompt injection?

By checking if the AI's answers contain unexpected or harmful content, so suspicious outputs can be caught and handled safely.

What is the main goal of prompt injection defense?

A Prevent AI from being tricked by harmful instructions

B Make AI respond faster

C Improve AI’s creativity

D Reduce AI’s training time

Prompt injection defense aims to stop harmful or unexpected instructions from tricking the AI.

Which method helps keep user input from changing AI instructions?

A Batch normalization

B Data augmentation

C Model pruning

D Context isolation

Context isolation separates user input from AI instructions to prevent prompt injection.

What should you do with suspicious user input before sending it to the AI?

A Filter or clean it

B Ignore it

C Add more instructions

D Send it as is

Filtering or cleaning suspicious input helps prevent harmful instructions from reaching the AI.

Why monitor AI outputs in prompt injection defense?

A To reduce server load

B To speed up responses

C To catch unexpected or harmful responses

D To increase word count

Monitoring outputs helps detect if the AI was tricked into giving bad answers.

Prompt injection attacks mainly target which part of AI systems?

A Model weights

B Input prompts

C Training data

D Hardware

Prompt injection attacks add harmful instructions into the input prompts.

Explain what prompt injection is and why it is important to defend against it.

Think about how someone might trick an AI with hidden commands.

Describe at least two ways to defend AI systems from prompt injection attacks.

Consider what you can do before and after the AI processes input.