Conditional generation in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Conditional generation

Which metric matters for Conditional Generation and WHY

In conditional generation, the model creates outputs based on given inputs or conditions. We want to check if the outputs are relevant and accurate for those conditions.

Common metrics include:

• BLEU score: Measures how close the generated text is to a reference text by comparing matching words or phrases.
• ROUGE score: Focuses on recall of overlapping words or phrases, useful for summarization tasks.
• Perplexity: Shows how well the model predicts the next word; lower is better.
• Accuracy: For tasks with discrete labels, measures correct predictions.

These metrics help us know if the model is generating outputs that fit the condition well and make sense.

Confusion Matrix or Equivalent Visualization

For conditional generation, confusion matrices are less common because outputs are often text or sequences, not fixed classes.

Instead, we use example comparisons:

Reference: "The cat sits on the mat."
Generated: "The cat is sitting on the mat."

BLEU score: 0.85 (high overlap)
Perplexity: 12.3 (low, good prediction)
    

This shows the model output is close to the reference, indicating good conditional generation.

Precision vs Recall Tradeoff with Examples

In conditional generation, precision means how many generated words are correct, recall means how many reference words are captured.

Example:

• High precision, low recall: Generated text is very accurate but misses some important details.
• High recall, low precision: Generated text covers many reference words but includes irrelevant or wrong words.

We want a balance so the output is both accurate and complete.

What "Good" vs "Bad" Metric Values Look Like

Good conditional generation metrics:

• BLEU score above 0.7 means strong similarity to reference.
• ROUGE scores above 0.6 indicate good coverage of important content.
• Low perplexity (e.g., below 20) means the model predicts words well.

Bad metrics:

• BLEU below 0.3 means poor match to reference.
• High perplexity (above 50) means model struggles to predict words.
• Outputs that do not follow the condition or are irrelevant.

Common Metrics Pitfalls

• Overfitting: Model memorizes training outputs, scoring high on metrics but failing on new inputs.
• Data leakage: Test data accidentally included in training, inflating metric scores.
• Metric mismatch: Using BLEU for very short or very different outputs can be misleading.
• Ignoring diversity: Metrics may favor safe, repetitive outputs over creative ones.

Self Check

Your conditional generation model has a BLEU score of 0.9 but generates very repetitive text that ignores some conditions. Is it good?

Answer: Not fully. High BLEU shows similarity to references, but ignoring conditions and repetition means the model is not truly generating condition-based diverse outputs. You should check other metrics and output quality.

Key Result

BLEU, ROUGE, and perplexity are key metrics to evaluate if conditional generation outputs match conditions well and are meaningful.

Practice

1. What is the main purpose of conditional generation in TensorFlow models?

easy

A. To generate outputs based on extra input information

B. To train models without any input data

C. To reduce the size of the model

D. To speed up the training process

Solution

1. Step 1: Understand conditional generation concept

   Conditional generation uses extra input information to guide the output generation.

2. Step 2: Identify the purpose in TensorFlow models

   This extra input helps models create outputs that depend on given conditions, like labels or attributes.

3. Final Answer:

   To generate outputs based on extra input information -> Option A

4. Quick Check:

   Conditional generation = guided output [OK]

Hint: Conditional means output depends on extra input [OK]

Common Mistakes:

• Thinking conditional generation removes input data
• Confusing conditional generation with model size reduction
• Assuming it only speeds up training

2. Which of the following is the correct way to combine noise and condition input in TensorFlow for conditional generation?

easy

A. tf.concat([noise, condition], axis=1)

B. tf.add(noise, condition)

C. tf.multiply(noise, condition)

D. tf.subtract(condition, noise)

Solution

1. Step 1: Understand input combination for conditional generation

   Noise and condition inputs are usually combined by concatenation along the feature axis.

2. Step 2: Identify correct TensorFlow function

   tf.concat joins tensors along a specified axis, which is the right way to combine noise and condition vectors.

3. Final Answer:

   tf.concat([noise, condition], axis=1) -> Option A

4. Quick Check:

   Combine inputs by concatenation [OK]

Hint: Use tf.concat to join inputs, not arithmetic ops [OK]

Common Mistakes:

• Using tf.add or tf.multiply which require same shapes
• Subtracting inputs which is not standard for combining
• Not specifying axis in concatenation

3. Given the code below, what will be the shape of the combined input tensor?

noise = tf.random.normal([32, 100])
condition = tf.random.normal([32, 10])
combined = tf.concat([noise, condition], axis=1)

medium

A. (32, 90)

B. (100, 10)

C. (32, 110)

D. (42, 110)

Solution

1. Step 1: Check shapes of noise and condition tensors

   Noise shape is (32, 100), condition shape is (32, 10).

2. Step 2: Understand concatenation along axis=1

   Concatenating along axis=1 adds the second dimension sizes: 100 + 10 = 110, batch size 32 stays same.

3. Final Answer:

   (32, 110) -> Option C

4. Quick Check:

   Concatenate dims: 100+10=110 [OK]

Hint: Concatenate adds sizes on axis=1, batch size unchanged [OK]

Common Mistakes:

• Adding batch sizes instead of feature sizes
• Mixing up axis parameter in concat
• Assuming shapes transpose automatically

4. Identify the error in this conditional generation code snippet:

noise = tf.random.normal([32, 100])
condition = tf.random.normal([10, 32])
combined = tf.concat([noise, condition], axis=1)

medium

A. Axis parameter should be 0

B. tf.concat cannot concatenate tensors

C. Noise tensor shape is invalid

D. Shapes of noise and condition do not match on axis 0

Solution

1. Step 1: Compare shapes of noise and condition tensors

   Noise shape is (32, 100), condition shape is (10, 32). Batch sizes differ (32 vs 10).

2. Step 2: Understand concat axis requirement

   Concatenation along axis=1 requires matching sizes on axis=0 (batch size). Here they differ, causing error.

3. Final Answer:

   Shapes of noise and condition do not match on axis 0 -> Option D

4. Quick Check:

   Batch sizes must match for concat [OK]

Hint: Check batch sizes match before concatenation [OK]

Common Mistakes:

• Assuming tf.concat works with any shapes
• Changing axis without fixing shape mismatch
• Ignoring batch size dimension

5. You want to build a conditional GAN in TensorFlow that generates images of specific digits (0-9). Which approach correctly integrates the digit label condition into the generator model?

hard

A. Add the label as a separate input to the discriminator only

B. Concatenate a one-hot encoded label vector with noise before feeding to the generator

C. Use the label to filter generated images after generation

D. Train the generator without any label input

Solution

1. Step 1: Understand conditional GAN input requirements

   The generator needs both noise and condition (digit label) to create images of specific digits.

2. Step 2: Identify correct integration method

   Concatenating one-hot encoded label with noise vector before input to generator allows conditioning generation on digit.

3. Final Answer:

   Concatenate a one-hot encoded label vector with noise before feeding to the generator -> Option B

4. Quick Check:

   Condition + noise input guides generation [OK]

Hint: Combine label and noise before generator input [OK]

Common Mistakes:

• Conditioning only discriminator, not generator
• Filtering outputs instead of conditioning input
• Ignoring label input in generator