Bird
Raised Fist0

Supervised vs Unsupervised vs Reinforcement Learning - When to Use Which

THEORYbeginnerInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
⚖️
Supervised Learning vs Unsupervised Learning

For a new dataset with unknown structure and no labels, would you start with supervised or unsupervised learning? Justify your choice.

📊
Comparison
DimensionSupervised LearningUnsupervised Learning
Data RequirementHigh (labeled data needed)Low (unlabeled data)
InterpretabilityModerate (depends on model)Low (hard to interpret)
ScalabilityHigh (with sufficient data)High (can handle large data)
Pattern DiscoveryLow (relies on labels)High (discovers patterns)
Evaluation EaseHigh (clear metrics)Low (no clear metrics)
✅
Verdict
Default to supervised learning when you have labeled data and a clear task. Switch to unsupervised learning when labels are unavailable or when exploring data structure is the primary goal.
💡
Key Insight
The choice between supervised and unsupervised learning hinges on label availability and the clarity of the task definition.
⚠️
Interview Trap
❌ Wrong: Always start with supervised learning because it provides clear results.
This ignores the scenario where no labels are available, making supervised learning impractical.
✅ Consider the availability of labels and the goal of the analysis. If labels are unavailable, start with unsupervised learning to explore data structure.
Supervised Learning
"Excels at tasks with clear, labeled data for prediction."
Weaknesses
  • Requires large amounts of labeled data, which can be costly to obtain.
  • Struggles with discovering hidden patterns without labels.
  • Overfitting risk if the model is too complex for the dataset size.
  • Limited to the scope of the labels provided.
Ideal When
  • n > 10K and labels are available and reliable
  • When the task is well-defined, such as classification or regression
  • When high accuracy on labeled data is critical
  • When you have a clear evaluation metric
Google uses supervised learning for image classification in Google Photos, where images are labeled with tags.
Unsupervised Learning
"Identifies hidden patterns and structures in unlabeled data."
Weaknesses
  • No direct way to evaluate the model's performance.
  • Results can be difficult to interpret and validate.
  • May require domain knowledge to make sense of clusters or patterns.
  • Can be sensitive to the choice of algorithm and parameters.
Ideal When
  • n > 50K and no labels are available
  • When exploring data to find natural groupings or patterns
  • When dimensionality reduction is needed for visualization
  • When preprocessing data for a subsequent supervised task
Spotify uses unsupervised learning for music recommendation by clustering users based on listening habits.
🐍
Supervised Learning - Example
▼ Show
Python
import numpy as np
from sklearn.linear_model import LogisticRegression
X = np.random.rand(100, 10)
y = np.random.randint(0, 2, 100)
model = LogisticRegression()
model.fit(X, y)
predictions = model.predict(X)
🐍
Unsupervised Learning - Example
▼ Show
Python
import numpy as np
from sklearn.cluster import KMeans
X = np.random.rand(100, 10)
kmeans = KMeans(n_clusters=3)
kmeans.fit(X)
clusters = kmeans.labels_
✅
Use Supervised Learning When
  • •n > 10K and you have labeled data
  • •When the task is classification or regression with clear metrics
  • •When you need high accuracy on labeled data
  • •When you have a clear evaluation metric
✅
Use Unsupervised Learning When
  • •n > 50K and no labels are available
  • •When exploring data to find natural groupings
  • •When dimensionality reduction is needed
  • •When preprocessing data for a subsequent supervised task
↳
Nuances
  • Unsupervised learning can be a precursor to supervised learning by providing insights into data structure.
  • Supervised learning is not always feasible if labeling is too costly or impractical.
  • Unsupervised learning can sometimes reveal unexpected insights that supervised learning might miss due to label bias.

Practice

(1/5)
1. 
import numpy as np
from sklearn.feature_selection import SelectKBest, f_classif
X = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
y = np.array([1, 0, 1])
selector = SelectKBest(score_func=f_classif, k=2)
X_new = selector.fit_transform(X, y)
print(X_new)
What does the above code output?
easy
A. array([[2, 3], [5, 6], [8, 9]])
B. array([[1, 3], [4, 6], [7, 9]])
C. array([[1, 2], [4, 5], [7, 8]])
D. array([[3, 2], [6, 5], [9, 8]])

Solution

  1. Step 1: Understand SelectKBest.

    SelectKBest selects the top k features based on the highest scores from the scoring function.

  2. Step 2: Determine the output.

    f_classif scores features based on ANOVA F-value. Here, features 2 and 3 have the highest scores.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Top k features by score. [OK]
Hint: SelectKBest picks top k features by score. [OK]
Common Mistakes:
2. A model trained on a dataset with engineered features is underperforming on the validation set. What is the most likely cause?
medium
A. The model is too simple for the complexity of the features.
B. The features are over-engineered, causing overfitting.
C. The model has too many hyperparameters to tune.
D. The features are not normalized, leading to poor model performance.

Solution

  1. Step 1: Analyze the symptoms of underperformance.

    Underperformance on validation suggests overfitting or poor generalization.

  2. Step 2: Evaluate the role of feature engineering.

    Over-engineered features (C) can lead to overfitting, especially if they capture noise. Normalization (B) affects convergence but not generalization directly. Model complexity (A, D) is less likely the issue if features are the focus.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Over-engineered features often lead to overfitting. [OK]
Hint: Over-engineered features often lead to overfitting. [OK]
Common Mistakes:
3. You have a model that is overfitting. Which approach is more suitable to address this issue?
medium
A. Add dropout layers to the model.
B. Reduce the size of the training dataset.
C. Use a smaller learning rate.
D. Increase the model complexity by adding more layers.

Solution

  1. Step 1: Identify methods to combat overfitting

    Dropout is a regularization technique that helps prevent overfitting by randomly dropping units during training.

  2. Step 2: Eliminate wrong options

    Increase the model complexity by adding more layers. increases overfitting risk, Reduce the size of the training dataset. reduces data, worsening generalization, and Use a smaller learning rate. affects convergence speed, not overfitting directly.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Dropout reduces overfitting [OK]
Hint: Dropout reduces overfitting [OK]
Common Mistakes:
4. You are tasked with reducing the dimensionality of a dataset for a clustering task. The dataset contains both numerical and categorical features. Which approach should you take?
hard
A. Use UMAP after one-hot encoding categorical features.
B. Use t-SNE after encoding categorical features.
C. Use LDA to reduce dimensions based on class labels.
D. Use PCA directly on the dataset.

Solution

  1. Step 1: Consider the nature of the data

    Dimensionality reduction techniques like PCA and t-SNE require numerical input.

  2. Step 2: Evaluate the options

    PCA cannot handle categorical data directly. t-SNE is computationally expensive and not ideal for mixed data types. LDA is not suitable for unsupervised clustering tasks.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    UMAP for mixed data types [OK]
Hint: UMAP for mixed data types [OK]
Common Mistakes:
5. You are tasked with designing a system to detect fraudulent transactions. Which approach is most appropriate given the need for adaptability to new fraud patterns?
hard
A. Use a rule-based system with predefined fraud patterns.
B. Rely on manual inspection of transactions.
C. Use a static decision tree with fixed thresholds.
D. Implement a machine learning model that can be retrained with new data.

Solution

  1. Step 1: Consider the need for adaptability.

    Fraud patterns change over time, requiring a system that can learn and adapt.

  2. Step 2: Evaluate the options for adaptability.

    Only a machine learning model that can be retrained offers the necessary adaptability.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    ML adapts, rules are static. [OK]
Hint: ML adapts, rules are static. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard