Text-to-image prompt crafting in Prompt Engineering / GenAI - Model Metrics & Evaluation

Metrics & Evaluation - Text-to-image prompt crafting

Which metric matters for text-to-image prompt crafting and WHY

For text-to-image models, the key metrics focus on how well the generated image matches the prompt. Common metrics include CLIP score, which measures similarity between the text prompt and the image, and FID (Fréchet Inception Distance), which measures image quality and diversity compared to real images. These metrics matter because they tell us if the prompt leads to images that are both relevant and visually realistic.

Confusion matrix or equivalent visualization

Text-to-image generation does not use a confusion matrix like classification. Instead, we use similarity scores. For example, a CLIP score ranges from 0 to 1, where higher means better match between prompt and image.

Prompt: "A red apple on a wooden table"
Generated Image CLIP score: 0.85 (high similarity)

Prompt: "A blue car in the forest"
Generated Image CLIP score: 0.45 (low similarity)

Precision vs Recall tradeoff with concrete examples

In text-to-image, think of precision as how accurately the image matches the prompt details, and recall as how well the image covers all aspects of the prompt.

Example:

• High precision, low recall: The image shows a red apple but misses the wooden table.
• Low precision, high recall: The image has a table and something red, but it is not clearly an apple.

Good prompt crafting aims to balance both, so the image is accurate and complete.

What "good" vs "bad" metric values look like for text-to-image prompt crafting

Good: CLIP score above 0.75, FID score low (closer to 0), and images clearly show prompt details.

Bad: CLIP score below 0.5, high FID score, images are blurry, unrelated, or miss key prompt elements.

Metrics pitfalls

• Overfitting: Model may generate images that look good on training prompts but fail on new prompts.
• Data leakage: If test prompts are too similar to training data, metrics may be misleadingly high.
• Accuracy paradox: High CLIP score does not always mean good image quality; images can match text but be unrealistic.
• Ignoring diversity: Low FID means images are realistic but may lack variety, causing repetitive outputs.

Self-check question

Your text-to-image model has a CLIP score of 0.9 but the images are blurry and lack detail. Is this good? Why or why not?

Answer: Not necessarily good. A high CLIP score means the image matches the prompt text, but blurriness and lack of detail show poor image quality. You need to improve image clarity and realism, not just text-image similarity.

Key Result

CLIP score and FID are key metrics to evaluate how well images match prompts and their quality.

Practice

1. What is the main purpose of crafting a text-to-image prompt?

easy

A. To describe what image you want the AI to create

B. To write code for training the AI model

C. To edit images after they are generated

D. To choose colors manually in the image

Solution

1. Step 1: Understand the role of a prompt

   A prompt is a description that tells the AI what image to make.

2. Step 2: Identify the correct purpose

   Only To describe what image you want the AI to create matches this role by describing the desired image.

3. Final Answer:

   To describe what image you want the AI to create -> Option A

4. Quick Check:

   Prompt = Image description [OK]

Hint: Prompts tell AI what to draw, not how to code [OK]

Common Mistakes:

• Confusing prompt with coding instructions
• Thinking prompt edits images directly
• Assuming prompt sets colors manually

2. Which of the following is the correct way to write a prompt for a text-to-image AI?

easy

A. def create_image(): return 'beach'

B. "A sunny beach with palm trees and clear blue water"

C.

D. SELECT * FROM images WHERE type='beach'

Solution

1. Step 1: Identify prompt format

   Prompts are plain text descriptions, not code or HTML.

2. Step 2: Match the correct option

   "A sunny beach with palm trees and clear blue water" is a clear text description suitable as a prompt.

3. Final Answer:

   "A sunny beach with palm trees and clear blue water" -> Option B

4. Quick Check:

   Prompt = Plain text description [OK]

Hint: Prompts are simple text, not code or tags [OK]

Common Mistakes:

• Using code or HTML instead of text
• Confusing prompts with programming functions
• Trying to query images with SQL as prompt

3. Given the prompt "A red apple on a wooden table, photorealistic style", what kind of image will the AI most likely generate?

medium

A. A cartoon apple with bright colors

B. A blurry sketch of an apple

C. A detailed, realistic photo of a red apple on wood

D. A text-only image with the words 'red apple'

Solution

1. Step 1: Analyze prompt details

   The prompt says "photorealistic style" and describes a red apple on a wooden table.

2. Step 2: Match prompt to image type

   The AI will generate a detailed, realistic photo-like image matching the description.

3. Final Answer:

   A detailed, realistic photo of a red apple on wood -> Option C

4. Quick Check:

   Photorealistic prompt = Realistic image [OK]

Hint: Look for style words like 'photorealistic' to guess output [OK]

Common Mistakes:

• Ignoring style words and expecting cartoons
• Confusing text prompts with text images
• Assuming blurry or sketch style without prompt

4. You wrote the prompt "A futuristic cityscape at night, neon lights, cyberpunk style" but the AI generated a daytime image without neon colors. What is the likely problem?

medium

A. The prompt lacks style details

B. The AI model ignored the style keywords

C. The prompt is too short and unclear

D. The prompt should specify 'night' and 'neon' more clearly

Solution

1. Step 1: Check prompt clarity

   The prompt mentions 'night' and 'neon lights' but may not emphasize them enough for the AI.

2. Step 2: Improve prompt specificity

   Adding stronger emphasis or repeating keywords helps AI focus on night and neon colors.

3. Final Answer:

   The prompt should specify 'night' and 'neon' more clearly -> Option D

4. Quick Check:

   Clear, strong keywords = better AI focus [OK]

Hint: Be very clear and repeat key style words in prompts [OK]

Common Mistakes:

• Assuming AI always understands subtle style hints
• Not emphasizing important details enough
• Blaming AI model instead of prompt clarity

5. You want to create a unique image of a "cat astronaut exploring Mars" with a watercolor painting style. Which prompt will most likely produce the best result?

hard

A. "A cat astronaut on Mars, watercolor painting, soft colors, detailed background"

B. "A cat on Earth, digital art style, bright colors"

C. "An astronaut on Mars, oil painting style, no animals"

D. "A dog astronaut exploring space, cartoon style"

Solution

1. Step 1: Match subject and style

   "A cat astronaut on Mars, watercolor painting, soft colors, detailed background" includes the cat astronaut, Mars setting, and watercolor style as requested.

2. Step 2: Check other options

   Options B, C, and D miss key elements like the cat, Mars, or watercolor style.

3. Final Answer:

   "A cat astronaut on Mars, watercolor painting, soft colors, detailed background" -> Option A

4. Quick Check:

   Complete, clear prompt = best image [OK]

Hint: Include all key subjects and style words clearly in prompt [OK]

Common Mistakes:

• Leaving out main subject or style
• Mixing up animals or settings
• Using vague or unrelated descriptions