Bird
Raised Fist0

Regression Metrics - MAE vs MSE

THEORYbeginnerInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
⚖️
MAE vs MSE

When evaluating a regression model, would you choose MAE or MSE and why?

📊
Comparison
DimensionMAEMSE
Sensitivity to outliersHighLow
InterpretabilityHighMedium
Error penalizationLowHigh
Optimization compatibilityMediumHigh
Cost structure alignmentLinearQuadratic
✅
Verdict
Default to MAE when outliers are present and interpretability is crucial; switch to MSE when large errors are particularly costly and data is clean.
💡
Key Insight
Choose MAE for robustness to outliers and interpretability, MSE for sensitivity to large errors.
⚠️
Interview Trap
❌ Wrong: Always use MSE because it is the standard metric.
Ignoring the impact of outliers and the specific cost structure of errors.
✅ Consider the presence of outliers and the cost structure of errors before choosing between MAE and MSE.
MAE
"Robustness to outliers"
Weaknesses
  • Less sensitive to large errors
  • Non-differentiable at zero, complicating gradient-based optimization
  • May not reflect the true cost of errors in some domains
Ideal When
  • When outliers are present and should not overly influence the metric
  • When interpretability of error magnitude is crucial
  • When the cost of errors is linear
Airbnb uses MAE for pricing models where outliers (e.g., luxury properties) should not skew the error metric.
MSE
"Sensitivity to large errors"
Weaknesses
  • Highly sensitive to outliers
  • Can over-penalize large errors, skewing model evaluation
  • Less interpretable in terms of actual error magnitude
Ideal When
  • When large errors are particularly costly
  • When the data is relatively clean with few outliers
  • When quadratic error cost is appropriate
Google uses MSE in their search ranking models where large deviations in predictions can significantly impact user experience.
🐍
MAE - Example
▼ Show
Python
import numpy as np

y_true = np.array([3, -0.5, 2, 7])
y_pred = np.array([2.5, 0.0, 2, 8])
mae = np.mean(np.abs(y_true - y_pred))
print('MAE:', mae)
🐍
MSE - Example
▼ Show
Python
import numpy as np

y_true = np.array([3, -0.5, 2, 7])
y_pred = np.array([2.5, 0.0, 2, 8])
mse = np.mean((y_true - y_pred) ** 2)
print('MSE:', mse)
✅
Use MAE When
  • •When n > 10K and outliers are present
  • •When interpretability of error is crucial
  • •When the cost of errors is linear
✅
Use MSE When
  • •When n < 10K and data is clean
  • •When large errors are particularly costly
  • •When quadratic error cost is appropriate
↳
Nuances
  • MAE is often preferred in domains where interpretability is key, despite its optimization challenges.
  • MSE can be misleading in datasets with outliers, as it may suggest a worse model performance than reality.
  • In financial applications, MAE is often used due to its linear cost alignment with monetary loss.

Practice

(1/5)
1. Which regularization technique is most suitable for feature selection in a linear regression model?
easy
A. L1 Lasso
B. Elastic Net
C. L2 Ridge
D. Dropout

Solution

  1. Step 1: Identify the purpose of feature selection.

    L1 Lasso regularization can shrink some coefficients to zero, effectively selecting features.

  2. Step 2: Eliminate wrong options.

    L2 Ridge does not shrink coefficients to zero. Elastic Net combines L1 and L2 but is not as direct for feature selection. Dropout is not used in linear regression.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    L1 Lasso zeros coefficients for selection. [OK]
Hint: L1 Lasso zeros coefficients for selection. [OK]
Common Mistakes:
2. In the context of tokenization for LLMs, which method is known for using a probabilistic model to determine the most likely tokenization of a sentence?
easy
A. SentencePiece
B. WordPiece
C. Byte Pair Encoding (BPE)
D. Character-level tokenization

Solution

  1. Step 1: Identify the method using a probabilistic model.

    SentencePiece uses a probabilistic model to tokenize sentences.

  2. Step 2: Eliminate wrong options.

    BPE and WordPiece do not use probabilistic models; character-level tokenization is too granular.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    SentencePiece uses probabilities. [OK]
Hint: SentencePiece uses probabilities. [OK]
Common Mistakes:
3. A model using attention mechanisms is not providing expected interpretability insights. What is a likely cause?
medium
A. The model is overfitting the training data
B. The attention weights are not being visualized correctly
C. The model architecture is too simple
D. The dataset is too small for attention to be effective

Solution

  1. Step 1: Identify the role of attention.

    Attention mechanisms provide insights through attention weights.

  2. Step 2: Diagnose the issue.

    If insights are missing, it may be due to incorrect visualization of attention weights.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Check attention weight visualization for insights. [OK]
Hint: Check attention weight visualization for insights. [OK]
Common Mistakes:
4. A Naive Bayes model is underperforming on a dataset. What is the most likely cause?
medium
A. The features are highly correlated.
B. The features are not normalized.
C. The model has too many parameters.
D. The dataset is too small.

Solution

  1. Step 1: Identify Potential Causes

    Naive Bayes assumes feature independence, so correlation can degrade performance.

  2. Step 2: Eliminate Other Causes

    The model has too many parameters. is incorrect because Naive Bayes is not parameter-heavy. The features are not normalized. is incorrect because normalization is not critical for Naive Bayes. The dataset is too small. is incorrect because Naive Bayes can work well with small datasets.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Correlated features degrade Naive Bayes. [[check]]
Hint: Correlated features degrade Naive Bayes. [[check]]
Common Mistakes:
5. You are designing a recommendation system for a video platform. Which metric would best capture user satisfaction if users tend to watch only the first few recommended videos?
hard
A. Precision at K
B. Mean Reciprocal Rank (MRR)
C. Normalized Discounted Cumulative Gain (NDCG)
D. Mean Average Precision (MAP)

Solution

  1. Step 1: Understand user behavior

    Users watching only the first few videos means early precision is crucial.

  2. Step 2: Choose the metric

    Precision at K directly measures the quality of the top K recommendations, aligning with user behavior.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Precision at K focuses on top results. [OK]
Hint: Precision at K focuses on top results. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard