Zero-shot prompting in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Zero-shot Prompting

What does zero-shot prompting mean in the context of AI language models?

A The model is given examples of the task during training and uses them to answer.

B The model is asked to perform a task without any prior examples or task-specific training.

C The model is fine-tuned on a small dataset before answering the prompt.

D The model generates answers based on random guesses without understanding.

💡 Hint

Think about how the model handles tasks without seeing examples first.

Zero-shot prompting means the model tries to answer a task without any examples or fine-tuning on that task. It relies on its general knowledge.

❓ Predict Output

intermediate

Output of Zero-shot Prompting Example

Given the prompt to a language model: "Translate 'Hello' to French." What is the most likely output?

Prompt Engineering / GenAI

prompt = "Translate 'Hello' to French."
# Model generates output based on zero-shot prompting

A "Bonjour"

B "Hello"

C "Hola"

D "Ciao"

💡 Hint

Think about the French word for 'Hello'.

The model uses its knowledge to translate 'Hello' to French, which is 'Bonjour'.

❓ Model Choice

advanced

Choosing a Model for Zero-shot Prompting

Which type of AI model is best suited for zero-shot prompting tasks?

A A large pre-trained language model trained on diverse data.

B A model trained only on images without text data.

C A small model trained only on a specific task dataset.

D A model fine-tuned on a single domain-specific dataset.

💡 Hint

Consider which model has broad knowledge to handle new tasks without examples.

Large pre-trained language models trained on diverse data can generalize well to new tasks without examples, making them ideal for zero-shot prompting.

❓ Hyperparameter

advanced

Effect of Temperature in Zero-shot Prompting

In zero-shot prompting, what effect does increasing the temperature parameter have on the model's output?

A It increases the model's training speed.

B It makes the output more deterministic and repetitive.

C It makes the output more random and diverse.

D It reduces the length of the output.

💡 Hint

Think about how temperature controls randomness in text generation.

Higher temperature increases randomness, making the model's output more diverse and less predictable.

❓ Metrics

expert

Evaluating Zero-shot Prompting Performance

Which metric is most appropriate to evaluate the quality of zero-shot prompting on a text classification task?

A Mean Squared Error - average squared difference between predictions and true values.

B Perplexity - measures how well the model predicts the next word.

C BLEU score - measures similarity between generated and reference text.

D Accuracy - percentage of correct predictions.

💡 Hint

Consider the type of task and what metric measures correct classification.

Accuracy is the best metric for classification tasks as it measures the percentage of correct predictions.