DataParallel basics in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - DataParallel basics

Which metric matters for DataParallel and WHY

When using DataParallel in PyTorch, the main goal is to speed up training by using multiple GPUs. The key metric to watch is training throughput, which means how many samples your model processes per second. This shows if DataParallel is helping your model train faster. Also, keep an eye on model accuracy or loss to make sure splitting data across GPUs does not harm learning.

Confusion matrix or equivalent visualization

DataParallel itself does not change prediction results or confusion matrix. But here is a simple confusion matrix example for a classification model trained with DataParallel:

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP): 80 | False Negative (FN): 20 |
      | False Positive (FP): 10 | True Negative (TN): 90 |
    

Totals: TP + FP + TN + FN = 200 samples. This matrix helps calculate precision and recall to check model quality.

Precision vs Recall tradeoff with examples

DataParallel speeds up training but does not directly affect precision or recall. However, if DataParallel causes bugs or synchronization issues, model quality might drop.

Example:

• High precision needed: Spam filter should avoid marking good emails as spam. DataParallel helps train faster but keep precision high.
• High recall needed: Cancer detection must find all cancer cases. DataParallel should not reduce recall by causing training errors.

What "good" vs "bad" metric values look like for DataParallel

Good:

• Training throughput increases significantly (e.g., 2x speed on 2 GPUs).
• Model accuracy and loss remain similar to single GPU training.
• No errors or crashes during training.

Bad:

• Training speed does not improve or gets slower.
• Model accuracy drops noticeably.
• Errors like mismatched tensor sizes or synchronization problems.

Common pitfalls with DataParallel metrics

• Accuracy paradox: Faster training but model quality drops unnoticed if only speed is checked.
• Data leakage: Incorrect data splitting across GPUs can leak test data into training.
• Overfitting indicators: Faster training might cause overfitting if validation metrics are ignored.
• GPU memory imbalance: Unequal data splits cause some GPUs to be idle, reducing speed gains.

Self-check question

Your model trained with DataParallel shows 98% accuracy but only 12% recall on fraud detection. Is it good for production? Why or why not?

Answer: No, it is not good. Even though accuracy is high, recall is very low. This means the model misses most fraud cases, which is dangerous. For fraud detection, high recall is critical to catch as many frauds as possible. You should improve recall before using the model in production.

Key Result

DataParallel should increase training speed without hurting model accuracy or recall.