Zero-shot prompting in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Zero-shot Prompting

What does zero-shot prompting mean in the context of AI language models?

A The model is given examples of the task during training and uses them to answer.

B The model is asked to perform a task without any prior examples or task-specific training.

C The model is fine-tuned on a small dataset before answering the prompt.

D The model generates answers based on random guesses without understanding.

💡 Hint

Think about how the model handles tasks without seeing examples first.

Zero-shot prompting means the model tries to answer a task without any examples or fine-tuning on that task. It relies on its general knowledge.

❓ Predict Output

intermediate

Output of Zero-shot Prompting Example

Given the prompt to a language model: "Translate 'Hello' to French." What is the most likely output?

Prompt Engineering / GenAI

prompt = "Translate 'Hello' to French."
# Model generates output based on zero-shot prompting

A "Bonjour"

B "Hello"

C "Hola"

D "Ciao"

💡 Hint

Think about the French word for 'Hello'.

The model uses its knowledge to translate 'Hello' to French, which is 'Bonjour'.

❓ Model Choice

advanced

Choosing a Model for Zero-shot Prompting

Which type of AI model is best suited for zero-shot prompting tasks?

A A large pre-trained language model trained on diverse data.

B A model trained only on images without text data.

C A small model trained only on a specific task dataset.

D A model fine-tuned on a single domain-specific dataset.

💡 Hint

Consider which model has broad knowledge to handle new tasks without examples.

Large pre-trained language models trained on diverse data can generalize well to new tasks without examples, making them ideal for zero-shot prompting.

❓ Hyperparameter

advanced

Effect of Temperature in Zero-shot Prompting

In zero-shot prompting, what effect does increasing the temperature parameter have on the model's output?

A It increases the model's training speed.

B It makes the output more deterministic and repetitive.

C It makes the output more random and diverse.

D It reduces the length of the output.

💡 Hint

Think about how temperature controls randomness in text generation.

Higher temperature increases randomness, making the model's output more diverse and less predictable.

❓ Metrics

expert

Evaluating Zero-shot Prompting Performance

Which metric is most appropriate to evaluate the quality of zero-shot prompting on a text classification task?

A Mean Squared Error - average squared difference between predictions and true values.

B Perplexity - measures how well the model predicts the next word.

C BLEU score - measures similarity between generated and reference text.

D Accuracy - percentage of correct predictions.

💡 Hint

Consider the type of task and what metric measures correct classification.

Accuracy is the best metric for classification tasks as it measures the percentage of correct predictions.

Practice

1. What is the main idea behind zero-shot prompting in AI?

easy

A. Training a model with many examples before testing

B. Fine-tuning a model with labeled data

C. Using a model only for image recognition tasks

D. Asking a model to perform a task using only instructions without examples

Solution

1. Step 1: Understand zero-shot prompting concept

   Zero-shot prompting means giving a model instructions to do a task without providing example inputs or outputs.

2. Step 2: Compare options to definition

   Only Asking a model to perform a task using only instructions without examples matches this idea. Options A, C, and D describe other AI methods.

3. Final Answer:

   Asking a model to perform a task using only instructions without examples -> Option D

4. Quick Check:

   Zero-shot prompting = instructions only [OK]

Hint: Zero-shot means no examples, just instructions [OK]

Common Mistakes:

• Confusing zero-shot with training on examples
• Thinking zero-shot needs fine-tuning
• Assuming zero-shot only works for images

2. Which of the following is the correct way to write a zero-shot prompt for a model to translate English to Spanish?

easy

A. "Translate the following sentence to Spanish: 'Hello, how are you?'"

B. "Here are examples: 'Hello' -> 'Hola', 'Goodbye' -> 'Adiós'. Translate 'Hello, how are you?'"

C. "Train the model with English-Spanish pairs before translating."

D. "Translate using a dictionary lookup for each word."

Solution

1. Step 1: Identify zero-shot prompt style

   Zero-shot prompts give instructions without examples or training data.

2. Step 2: Check options for instructions only

   "Translate the following sentence to Spanish: 'Hello, how are you?'" is a direct instruction without examples. "Here are examples: 'Hello' -> 'Hola', 'Goodbye' -> 'Adiós'. Translate 'Hello, how are you?'" includes examples, so it's not zero-shot. Options C and D describe other methods.

3. Final Answer:

   "Translate the following sentence to Spanish: 'Hello, how are you?'" -> Option A

4. Quick Check:

   Zero-shot prompt = instruction only [OK]

Hint: Zero-shot prompts have no examples, just clear instructions [OK]

Common Mistakes:

• Including examples in zero-shot prompts
• Confusing zero-shot with few-shot prompting
• Thinking training is needed for zero-shot

3. Given this zero-shot prompt to a language model:
"Summarize this text in one sentence: 'The cat sat on the mat because it was tired.'"
What is the most likely model output?

medium

A. "Because it was tired, the cat sat on the mat, and the dog barked."

B. "The cat sat on the mat."

C. "The cat was tired and sat on the mat."

D. ""

Solution

1. Step 1: Understand the prompt and task

   The prompt asks for a one-sentence summary of the given text.

2. Step 2: Evaluate options for correct summary

   "The cat was tired and sat on the mat." captures the main idea clearly and concisely. "The cat sat on the mat." is incomplete, missing the reason. "Because it was tired, the cat sat on the mat, and the dog barked." adds unrelated info. "" is empty, so invalid.

3. Final Answer:

   "The cat was tired and sat on the mat." -> Option C

4. Quick Check:

   Summary includes main points = "The cat was tired and sat on the mat." [OK]

Hint: Summaries keep main ideas, no extra details [OK]

Common Mistakes:

• Choosing incomplete or unrelated outputs
• Ignoring the instruction to summarize in one sentence
• Selecting empty or irrelevant answers

4. You wrote this zero-shot prompt:
"Explain the benefits of exercise"
But the model returns an error or unrelated text. What is the likely problem?

medium

A. The prompt is too vague or lacks clear instructions

B. The model requires example inputs and outputs

C. The prompt uses too many examples

D. The model cannot understand English

Solution

1. Step 1: Analyze the prompt clarity

   The prompt "Explain the benefits of exercise" is short but may be too vague or lacks detail for the model to respond well.

2. Step 2: Consider model requirements

   Zero-shot prompting works best with clear, simple instructions. The model does not require examples (so B is wrong). The prompt has no examples (so C is wrong). The model understanding English is assumed (A is unlikely).

3. Final Answer:

   The prompt is too vague or lacks clear instructions -> Option A

4. Quick Check:

   Clear instructions needed for zero-shot [OK]

Hint: Make prompts clear and specific to avoid errors [OK]

Common Mistakes:

• Assuming examples are always needed
• Ignoring prompt clarity
• Blaming model language understanding incorrectly

5. You want to use zero-shot prompting to classify customer reviews as positive or negative. Which prompt is best to get accurate results?

hard

A. "Train a model on labeled reviews before classifying."

B. "Classify this review as positive or negative: 'The product works great and arrived on time.'"

C. "Here are examples: 'Good' -> positive, 'Bad' -> negative. Classify: 'The product works great and arrived on time.'"

D. "Translate the review to another language before classifying."

Solution

1. Step 1: Identify zero-shot prompt requirements

   Zero-shot prompting uses instructions only, no examples or training.

2. Step 2: Evaluate prompt options

   "Classify this review as positive or negative: 'The product works great and arrived on time.'" is a clear instruction without examples, fitting zero-shot. "Here are examples: 'Good' -> positive, 'Bad' -> negative. Classify: 'The product works great and arrived on time.'" includes examples, so it's few-shot. "Train a model on labeled reviews before classifying." requires training, not zero-shot. "Translate the review to another language before classifying." is unrelated to classification.

3. Final Answer:

   "Classify this review as positive or negative: 'The product works great and arrived on time.'" -> Option B

4. Quick Check:

   Zero-shot = instruction only, no examples [OK]

Hint: Use clear instructions without examples for zero-shot tasks [OK]

Common Mistakes:

• Adding examples in zero-shot prompts
• Confusing zero-shot with training or few-shot
• Using unrelated steps like translation