Multi-input and multi-output models in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Multi-input and multi-output models

Which metric matters for Multi-input and Multi-output models and WHY

When a model has multiple inputs and outputs, each output can represent a different task. So, we need to measure performance separately for each output. For example, if one output predicts a category (classification), accuracy or F1-score matters. If another output predicts a number (regression), mean squared error (MSE) matters. This helps us understand how well the model does on each task.

Confusion matrix or equivalent visualization

For classification outputs, a confusion matrix shows how many predictions were correct or wrong for each class. For example, if one output predicts "cat" or "dog":

      | Predicted Cat | Predicted Dog |
      |--------------|---------------|
      | True Cat: 50 | False Dog: 3  |
      | False Cat: 5 | True Dog: 42  |
    

This helps calculate precision, recall, and F1 for that output. For regression outputs, we look at error values like MSE instead.

Precision vs Recall tradeoff with concrete examples

Imagine a multi-output model where one output detects spam emails (classification) and another predicts email length (regression). For spam detection, high precision means few good emails are wrongly marked as spam. High recall means most spam emails are caught. Depending on what matters more, we adjust the model or threshold.

For the length prediction output, we focus on minimizing error, not precision or recall.

What "good" vs "bad" metric values look like for this use case

Good metrics mean each output performs well on its task. For classification outputs, precision and recall above 0.8 are usually good. For regression outputs, low MSE or MAE (mean absolute error) is good.

Bad metrics are low precision or recall (below 0.5) for classification, or very high error for regression. This means the model struggles on that output.

Common pitfalls in metrics for multi-input and multi-output models

• Ignoring some outputs: Only checking metrics for one output hides problems in others.
• Mixing metric types: Using accuracy for regression outputs or MSE for classification is wrong.
• Data leakage: If inputs share information that leaks target info, metrics look too good.
• Overfitting: High training metrics but poor validation metrics on any output means overfitting.

Self-check question

Your multi-output model has 98% accuracy on one classification output but only 12% recall on detecting fraud in another output. Is it good for production? Why or why not?

Answer: No, it is not good. Even though accuracy is high on one output, the very low recall on fraud detection means the model misses most fraud cases. For fraud detection, recall is critical because missing fraud is costly. So, the model needs improvement on that output before production.

Key Result

Evaluate each output separately using the right metric type; high recall is critical for sensitive tasks like fraud detection.