nn.LSTM layer in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - nn.LSTM layer

Which metric matters for nn.LSTM layer and WHY

The nn.LSTM layer is used for sequence data like text or time series. The main goal is to predict sequences or classify them correctly. So, metrics like accuracy for classification or mean squared error (MSE) for regression matter most. For classification, accuracy tells how many sequences were predicted right. For regression, MSE shows how close predictions are to true values. These metrics help us know if the LSTM learned useful patterns over time steps.

Confusion matrix example for nn.LSTM classification

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    50    |   10
      Negative           |    5     |   35

      Total samples = 50 + 10 + 5 + 35 = 100

      Precision = TP / (TP + FP) = 50 / (50 + 5) = 0.91
      Recall = TP / (TP + FN) = 50 / (50 + 10) = 0.83
      Accuracy = (TP + TN) / Total = (50 + 35) / 100 = 0.85
    

This confusion matrix shows how well the LSTM classified sequences. TP means correct positive predictions, FP means wrong positive predictions, and so on.

Precision vs Recall tradeoff for nn.LSTM

Imagine an LSTM model detecting spam emails (sequence classification). If it has high precision, it means most emails marked as spam really are spam. This avoids annoying users by wrongly blocking good emails.

If it has high recall, it finds almost all spam emails, but might mark some good emails as spam (false alarms).

Depending on what matters more, you tune the LSTM to balance precision and recall. For spam, high precision is often preferred to avoid blocking good mail.

Good vs Bad metric values for nn.LSTM

Good: Accuracy above 85% for classification, precision and recall both above 80%, and low MSE for regression tasks.

Bad: Accuracy near random guess (e.g., 50% for binary), very low recall (missing many true cases), or very high MSE showing poor predictions.

Good metrics mean the LSTM learned useful sequence patterns. Bad metrics mean it failed to capture time dependencies or overfitted.

Common pitfalls in metrics for nn.LSTM

• Accuracy paradox: High accuracy but poor recall if data is imbalanced (e.g., rare events).
• Data leakage: If future time steps leak into training, metrics look unrealistically good.
• Overfitting: Training metrics very good but validation metrics poor, meaning LSTM memorized sequences instead of generalizing.
• Ignoring sequence length: Metrics averaged over sequences of different lengths can be misleading.

Self-check question

Your LSTM model has 98% accuracy but only 12% recall on fraud detection sequences. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall means the model misses most fraud cases, which is dangerous. Even with high accuracy, missing fraud is costly. You should improve recall before using it in production.

Key Result

For nn.LSTM, accuracy and recall are key metrics; high recall is critical in tasks like fraud detection to avoid missing important cases.