Bird
Raised Fist0

How to Frame a Business Problem as an ML Problem

THEORYintermediateInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
⚖️
Framing as a Classification Problem vs Framing as a Regression Problem

Given a business problem with historical data, would you frame it as a classification or regression problem? Justify your choice.

📊
Comparison
DimensionFraming as a Classification ProblemFraming as a Regression Problem
Outcome TypeDiscreteContinuous
InterpretabilityHighMedium
Data RequirementsLowHigh
Sensitivity to OutliersLowHigh
Computational ComplexityLowHigh
✅
Verdict
Default to classification for problems with discrete outcomes and clear decision boundaries. Switch to regression when precision in continuous outcomes is necessary and data volume supports complex modeling.
💡
Key Insight
The choice between classification and regression hinges on the nature of the outcome and the business need for precision versus interpretability.
⚠️
Interview Trap
❌ Wrong: Always use classification for any business problem with discrete outcomes.
Ignores the potential need for precision in continuous outcomes or the presence of nuanced patterns.
✅ Consider the nature of the outcome and the business context, evaluating whether precision or interpretability is more critical.
Framing as a Classification Problem
"Excels in scenarios where outcomes are discrete and well-defined."
Weaknesses
  • Limited to discrete outcomes, unsuitable for continuous predictions.
  • May oversimplify problems with nuanced outcomes.
  • Requires balanced class distribution for optimal performance.
  • Struggles with high-dimensional feature spaces without feature engineering.
Ideal When
  • When the outcome is binary or categorical (e.g., n < 10K, classes < 10).
  • When interpretability of decision boundaries is crucial.
  • When the business decision is threshold-based (e.g., fraud detection).
  • When class imbalance can be managed effectively.
Facebook uses classification to detect spam posts, where the outcome is binary: spam or not spam.
Framing as a Regression Problem
"Ideal for predicting continuous outcomes with precision."
Weaknesses
  • Assumes linearity unless modeled otherwise, which may not capture complex patterns.
  • Sensitive to outliers, which can skew predictions.
  • Requires large datasets to generalize well.
  • Can be computationally intensive with high-dimensional data.
Ideal When
  • When the outcome is continuous and requires precise estimation (e.g., n > 10K).
  • When the relationship between features and outcome is linear or can be linearized.
  • When the business problem involves forecasting or trend analysis.
  • When sufficient data is available to model complex relationships.
Amazon uses regression to predict delivery times based on historical data, where the outcome is continuous.
🐍
Framing as a Classification Problem - Example
▼ Show
Python
import numpy as np
from sklearn.linear_model import LogisticRegression
X = np.random.rand(100, 5)
y = np.random.randint(0, 2, 100)
model = LogisticRegression()
model.fit(X, y)
predictions = model.predict(X)
🐍
Framing as a Regression Problem - Example
▼ Show
Python
import numpy as np
from sklearn.linear_model import LinearRegression
X = np.random.rand(100, 5)
y = np.random.rand(100)
model = LinearRegression()
model.fit(X, y)
predictions = model.predict(X)
✅
Use Framing as a Classification Problem When
  • •When the outcome is naturally categorical (e.g., churn prediction).
  • •When interpretability of decision boundaries is necessary.
  • •When the dataset is small (n < 10K) and classes are balanced.
  • •When the business decision is binary or multi-class.
✅
Use Framing as a Regression Problem When
  • •When the outcome is continuous and precision is required.
  • •When the dataset is large (n > 10K) and complex relationships exist.
  • •When forecasting or trend analysis is the primary goal.
  • •When the problem involves predicting a numerical value.
↳
Nuances
  • For problems with mixed outcomes, consider hybrid models that combine classification and regression.
  • Classification can be used for regression problems by discretizing the outcome, but this may lose precision.
  • Regression models can handle classification by setting thresholds, but this may complicate interpretability.
  • In cases with imbalanced classes, classification may require additional techniques like resampling.

Practice

(1/5)
1. In a binary classification problem where false positives are costly, which metric should be prioritized?
easy
A. Precision
B. F1 Score
C. Recall
D. Accuracy

Solution

  1. Step 1: Identify the cost concern.

    False positives are costly, so we need to minimize them.

  2. Step 2: Choose the metric that minimizes false positives.

    Precision is the metric that focuses on minimizing false positives.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Precision minimizes false positives. [OK]
Hint: Precision minimizes false positives. [OK]
Common Mistakes:
2. In a dataset with missing values in a numerical feature, which imputation strategy is most appropriate when the data is normally distributed?
easy
A. Use median imputation
B. Use mode imputation
C. Use zero imputation
D. Use mean imputation

Solution

  1. Step 1: Identify the distribution of the data

    For normally distributed data, the mean is the best measure of central tendency.

  2. Step 2: Eliminate wrong options

    Median is better for skewed data, mode for categorical data, and zero imputation can distort the distribution.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Mean for normal distribution [OK]
Hint: Mean for normal distribution [OK]
Common Mistakes:
3. In the context of Random Forests, what is the primary purpose of using bagging?
easy
A. To increase the model's interpretability
B. To enhance the model's feature selection
C. To improve the model's computational efficiency
D. To reduce the model's variance

Solution

  1. Step 1: Understand bagging's role

    Bagging, or Bootstrap Aggregating, is used to reduce variance by averaging predictions from multiple models trained on different subsets of the data.

  2. Step 2: Eliminate wrong options

    To increase the model's interpretability is incorrect because bagging does not inherently increase interpretability. To enhance the model's feature selection is wrong as feature selection is not the primary goal of bagging. To improve the model's computational efficiency is incorrect because bagging typically increases computational cost due to multiple models.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Bagging reduces variance by averaging. [OK]
Hint: Bagging reduces variance by averaging. [OK]
Common Mistakes:
4. When comparing two models using statistical significance, which approach is more appropriate if you have a small sample size?
medium
A. Use a t-test assuming equal variances.
B. Use a non-parametric test like the Wilcoxon signed-rank test.
C. Use a z-test assuming normal distribution.
D. Use a chi-squared test for independence.

Solution

  1. Step 1: Consider sample size constraints

    With small sample sizes, non-parametric tests are more reliable as they do not assume normal distribution.

  2. Step 2: Eliminate inappropriate tests

    Use a t-test assuming equal variances. assumes equal variances which may not hold. Use a z-test assuming normal distribution. assumes normal distribution, which is unreliable with small samples. Use a chi-squared test for independence. is for categorical data, not model comparison.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Small samples? Use non-parametric tests [OK]
Hint: Small samples? Use non-parametric tests [OK]
Common Mistakes:
5. How would you adapt Naive Bayes for a text classification task where word order matters?
hard
A. Use a bag-of-words model.
B. Use a single Gaussian distribution for all features.
C. Incorporate n-grams into the feature set.
D. Apply PCA to reduce dimensionality.

Solution

  1. Step 1: Understand the Task Requirements

    Word order matters, so capturing sequences is important.

  2. Step 2: Evaluate Adaptation Options

    Use a bag-of-words model. ignores word order. Use a single Gaussian distribution for all features. is irrelevant to text data. Apply PCA to reduce dimensionality. reduces dimensionality but doesn't capture order. Incorporate n-grams into the feature set. captures sequences with n-grams.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Use n-grams for word order in text. [[check]]
Hint: Use n-grams for word order in text. [[check]]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard