BCELoss vs BCEWithLogitsLoss in PyTorch: Key Differences and Usage

In PyTorch, BCELoss expects input probabilities (values between 0 and 1), so you must apply a sigmoid activation before it. BCEWithLogitsLoss combines sigmoid activation and binary cross-entropy in one step, making it more stable and convenient for raw model outputs (logits).

Quick Comparison

This table summarizes the main differences between BCELoss and BCEWithLogitsLoss in PyTorch.

Aspect	BCELoss	BCEWithLogitsLoss
Input expected	Probabilities (0 to 1) after sigmoid	Raw logits (no sigmoid needed)
Includes sigmoid?	No, must apply sigmoid manually	Yes, built-in sigmoid activation
Numerical stability	Less stable, can cause gradient issues	More stable, recommended for logits
Typical use case	When input is already sigmoid output	When model outputs raw scores (logits)
Performance	Slightly slower due to separate sigmoid	Faster and more efficient
Common error	Input not in [0,1] causes wrong loss	No such error, handles raw inputs safely

Key Differences

BCELoss requires the input to be probabilities between 0 and 1, so you must apply a sigmoid function to your model's raw outputs before passing them to this loss. This extra step can lead to numerical instability, especially when probabilities are very close to 0 or 1, causing gradients to vanish or explode.

On the other hand, BCEWithLogitsLoss combines the sigmoid activation and binary cross-entropy loss into a single function. It takes raw logits directly from the model and applies a numerically stable sigmoid internally. This reduces the risk of numerical errors and improves training stability.

Because of this, BCEWithLogitsLoss is generally preferred for binary classification tasks in PyTorch. It simplifies the code by removing the need for a separate sigmoid layer and improves performance and accuracy by handling edge cases better.

Code Comparison

Here is how you use BCELoss by manually applying sigmoid to model outputs before computing the loss.

python

import torch
import torch.nn as nn

# Sample raw model output (logits)
logits = torch.tensor([0.2, -1.5, 3.0, 0.7])
# Target labels
targets = torch.tensor([1., 0., 1., 0.])

# Apply sigmoid manually
probabilities = torch.sigmoid(logits)

# Define BCELoss
criterion = nn.BCELoss()

# Compute loss
loss = criterion(probabilities, targets)
print(f"BCELoss: {loss.item():.4f}")

Output

BCELoss: 0.4147

BCEWithLogitsLoss Equivalent

Here is the equivalent code using BCEWithLogitsLoss which takes logits directly without sigmoid.

python

import torch
import torch.nn as nn

# Sample raw model output (logits)
logits = torch.tensor([0.2, -1.5, 3.0, 0.7])
# Target labels
targets = torch.tensor([1., 0., 1., 0.])

# Define BCEWithLogitsLoss
criterion = nn.BCEWithLogitsLoss()

# Compute loss directly on logits
loss = criterion(logits, targets)
print(f"BCEWithLogitsLoss: {loss.item():.4f}")

Output

BCEWithLogitsLoss: 0.4147

When to Use Which

Choose BCELoss only if your model already outputs probabilities (after sigmoid) or if you have a specific reason to separate sigmoid and loss.

Choose BCEWithLogitsLoss in almost all other cases, especially when your model outputs raw scores (logits). It is more stable, simpler to use, and less error-prone.

In practice, BCEWithLogitsLoss is the recommended default for binary classification tasks in PyTorch.

Key Takeaways

Use BCEWithLogitsLoss for raw model outputs (logits) to get stable and efficient training.

BCELoss requires manual sigmoid on inputs and is less numerically stable.

BCEWithLogitsLoss combines sigmoid and loss in one step, reducing errors.

Choose BCELoss only if inputs are already probabilities between 0 and 1.

BCEWithLogitsLoss is the preferred and safer choice for binary classification.