Image understanding and description in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the primary role of an image captioning model?

Imagine you have a smart assistant that looks at pictures and tells you what it sees in simple sentences. What is the main job of this assistant?

A To generate a short text description that explains the content of the image.

B To classify the image into one of many categories without describing it.

C To enhance the image quality by removing noise and improving colors.

D To detect faces in the image and blur them for privacy.

💡 Hint

Think about what it means to 'describe' an image in words.

An image captioning model looks at an image and produces a sentence or phrase that describes what is in the image. This helps people understand the image content through text.

❓ Predict Output

intermediate

What is the output of this image captioning code snippet?

Given the following simplified code that uses a pre-trained image captioning model, what will be printed?

Prompt Engineering / GenAI

image = load_image('dog_park.jpg')
caption = model.generate_caption(image)
print(caption)

A "A group of dogs playing in a park."

B SyntaxError: missing parentheses in call to 'print'

C "dog_park.jpg"

D None

💡 Hint

The model generates a text description of the image, not just the filename.

The code loads an image and uses the model to generate a caption describing the image content. The print statement outputs that caption.

❓ Model Choice

advanced

Which model architecture is best suited for image captioning tasks?

You want to build a system that looks at images and writes sentences describing them. Which model type is most appropriate?

A A simple feedforward neural network with no sequence handling.

B A convolutional neural network (CNN) combined with a recurrent neural network (RNN).

C A support vector machine (SVM) classifier.

D A k-means clustering algorithm.

💡 Hint

Think about how images and sentences are processed differently and how to combine them.

CNNs extract features from images, and RNNs generate sequences of words. Combining them allows the model to understand images and produce text descriptions.

❓ Metrics

advanced

Which metric is commonly used to evaluate image captioning quality?

After training an image captioning model, you want to measure how good its descriptions are compared to human-written captions. Which metric should you use?

A Confusion matrix of detected objects.

B Mean Squared Error (MSE) between pixel values of images.

C BLEU score, which compares the overlap of words and phrases between generated and reference captions.

D Accuracy of classifying images into categories.

💡 Hint

Think about metrics that compare text similarity.

BLEU score measures how closely the generated captions match human captions by comparing word sequences, making it suitable for caption quality evaluation.

🔧 Debug

expert

Why does this image captioning model produce repetitive captions?

Consider this simplified code snippet where the model generates captions but repeats the same word multiple times:

caption = model.generate_caption(image)
print(caption)
# Output: "dog dog dog dog dog"

What is the most likely cause?

A The print statement is inside a loop printing the same word multiple times.

B The input image is corrupted and cannot be processed.

C The model was trained on only one image, so it memorizes that caption.

D The model's beam search decoding is not implemented correctly, causing it to select the same word repeatedly.

💡 Hint

Think about how the model chooses words during caption generation.

If beam search or greedy decoding is faulty, the model may get stuck repeating the same word. This is a common decoding bug in sequence generation models.