One-vs-rest and one-vs-one strategies in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Difference between One-vs-Rest and One-vs-One

Which statement correctly describes the difference between the One-vs-Rest (OvR) and One-vs-One (OvO) strategies for multi-class classification?

A OvR trains classifiers for every pair of classes, while OvO trains one classifier per class against all others.

B OvR and OvO both train one classifier per class but use different loss functions.

C OvR uses a single classifier for all classes, while OvO uses multiple classifiers for each class.

D OvR trains one classifier per class against all other classes, while OvO trains classifiers for every pair of classes.

💡 Hint

Think about how many classifiers are trained in each strategy and what data they use.

One-vs-Rest trains one classifier per class, distinguishing that class from all others combined. One-vs-One trains a classifier for every pair of classes, focusing on distinguishing between just two classes at a time.

❓ Model Choice

intermediate

Choosing strategy for many classes

You have a dataset with 50 classes and limited computing power. Which multi-class classification strategy is generally more efficient to train?

A One-vs-Rest, because it trains fewer classifiers (one per class).

B One-vs-One, because it trains classifiers only on pairs of classes.

C Both strategies require the same training time regardless of class count.

D One-vs-One, because it requires less memory for large numbers of classes.

💡 Hint

Consider how the number of classifiers grows with the number of classes in each strategy.

One-vs-Rest trains one classifier per class, so with 50 classes it trains 50 classifiers. One-vs-One trains classifiers for every pair of classes, which is 50*49/2 = 1225 classifiers, much more computationally expensive.

❓ Predict Output

advanced

Output of OvR prediction voting

Given the following OvR classifiers' decision scores for a sample, what is the predicted class?

scores = {'class_A': -1.2, 'class_B': 0.5, 'class_C': 0.3}

OvR predicts the class with the highest decision score.

A class_B

B class_A

C class_C

D No prediction because scores are negative

💡 Hint

Look for the highest score regardless of sign.

OvR picks the class with the highest decision score. Here, class_B has the highest score 0.5, so it is predicted.

❓ Metrics

advanced

Calculating number of classifiers in OvO

For a multi-class problem with 7 classes, how many binary classifiers does the One-vs-One strategy train?

A 49

B 21

C 42

D 7

💡 Hint

Use the formula for combinations of 7 classes taken 2 at a time.

Number of classifiers in OvO is n*(n-1)/2 = 7*6/2 = 21.

🔧 Debug

expert

Error in OvR prediction code

What error will this Python code raise?

def predict_ovr(scores):
    # scores is a dict of class:score
    max_score = max(scores.values())
    for cls, score in scores.items():
        if score == max_score:
            return cls

result = predict_ovr({'A': 0.2, 'B': 0.2, 'C': 0.1})
print(result)

A Raises ValueError due to multiple max values.

B Returns 'B' because it finds the last max score match.

C Returns 'A' because it finds the first max score match.

D Raises TypeError because max() cannot be used on dict values.

💡 Hint

Consider how max() and the for loop behave with ties.

The code returns the first class with the max score. Since 'A' and 'B' tie at 0.2, it returns 'A'. No error is raised.

Practice

1. What is the main idea behind the one-vs-rest strategy in multi-class classification?

easy

A. Train one model per class to separate that class from all others combined.

B. Train one model for every pair of classes.

C. Train a single model to classify all classes at once.

D. Train models only for the most frequent classes.

Solution

1. Step 1: Understand one-vs-rest approach

   One-vs-rest means creating one model per class. Each model learns to separate its class from all other classes combined.

2. Step 2: Compare with other options

   One-vs-one trains models for every pair, not per class. Single model for all classes is not one-vs-rest. Training only on frequent classes is unrelated.

3. Final Answer:

   Train one model per class to separate that class from all others combined. -> Option A

4. Quick Check:

   One-vs-rest = One model per class [OK]

Hint: One-vs-rest means one model per class vs all others [OK]

Common Mistakes:

• Confusing one-vs-rest with one-vs-one
• Thinking one-vs-rest uses one model for all classes
• Assuming one-vs-rest trains only on frequent classes

2. Which of the following correctly describes the number of models trained in the one-vs-one strategy for a problem with 4 classes?

easy

A. 4 models

B. 6 models

C. 1 model

D. 8 models

Solution

1. Step 1: Calculate number of pairs for 4 classes

   One-vs-one trains a model for every pair of classes. Number of pairs = 4 choose 2 = 4*3/2 = 6.

2. Step 2: Verify other options

   4 models is one per class (one-vs-rest). 1 model is single multi-class. 8 models is incorrect count.

3. Final Answer:

   6 models -> Option B

4. Quick Check:

   Pairs for 4 classes = 6 [OK]

Hint: Number of one-vs-one models = n*(n-1)/2 [OK]

Common Mistakes:

• Using number of classes instead of pairs
• Confusing one-vs-one with one-vs-rest counts
• Calculating pairs incorrectly

3. Consider a dataset with 3 classes: A, B, and C. Using one-vs-rest, how many models are trained and what does each model learn?

medium

A. 6 models; each separates pairs of classes.

B. 3 models; each separates one class from one other class only.

C. 1 model; separates all three classes at once.

D. 3 models; each separates one class from the other two combined.

Solution

1. Step 1: Count models in one-vs-rest for 3 classes

   One-vs-rest trains one model per class, so 3 models total.

2. Step 2: Understand model learning in one-vs-rest

   Each model learns to separate its class from all other classes combined (not just one other class).

3. Final Answer:

   3 models; each separates one class from the other two combined. -> Option D

4. Quick Check:

   One-vs-rest with 3 classes = 3 models [OK]

Hint: One-vs-rest trains one model per class vs all others [OK]

Common Mistakes:

• Thinking one-vs-rest trains models per pair
• Assuming only one model is trained
• Confusing one-vs-rest with one-vs-one

4. You implemented one-vs-one for a 5-class problem but only trained 4 models. What is the likely mistake?

medium

A. You trained models only for the most frequent classes.

B. You trained one model per class instead of pairs.

C. You forgot to train models for all pairs; should be 10 models.

D. You trained a single multi-class model.

Solution

1. Step 1: Calculate expected number of one-vs-one models for 5 classes

   Number of pairs = 5 choose 2 = 5*4/2 = 10 models expected.

2. Step 2: Identify mistake from training only 4 models

   Training only 4 models means some pairs were missed. Possibly forgot to train all pairs.

3. Final Answer:

   You forgot to train models for all pairs; should be 10 models. -> Option C

4. Quick Check:

   One-vs-one for 5 classes = 10 models [OK]

Hint: One-vs-one needs n*(n-1)/2 models; check count [OK]

Common Mistakes:

• Counting models as number of classes
• Confusing one-vs-one with one-vs-rest
• Training incomplete pairs

5. You have a 4-class problem with unbalanced data. Which strategy is better to handle this imbalance and why?

hard

A. One-vs-one, because training on pairs reduces imbalance impact between classes.

B. Neither, use a single multi-class model only.

C. One-vs-rest, because each model focuses on separating one class from all others, helping with imbalance.

D. Train only on the largest class to simplify the problem.

Solution

1. Step 1: Understand imbalance effect on one-vs-rest

   One-vs-rest models separate one class vs all others combined, which can cause imbalance if one class is small and others are large.

2. Step 2: Understand one-vs-one advantage

   One-vs-one trains models on pairs of classes, so imbalance is less severe per model, improving learning on minority classes.

3. Step 3: Evaluate other options

   Single multi-class model may struggle with imbalance. Training only on largest class ignores others.

4. Final Answer:

   One-vs-one, because training on pairs reduces imbalance impact between classes. -> Option A

5. Quick Check:

   One-vs-one handles imbalance better [OK]

Hint: One-vs-one handles imbalance better by focusing on pairs [OK]

Common Mistakes:

• Assuming one-vs-rest always better for imbalance
• Ignoring imbalance effects on combined classes
• Choosing single model ignoring class distribution