Learning curves in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding the shape of learning curves

Which of the following best describes the typical shape of a learning curve for a well-trained machine learning model as the training data size increases?

A Both training and validation errors decrease and then plateau as data size grows.

B The training error increases and the validation error decreases steadily.

C Training error remains constant while validation error fluctuates randomly.

D Training error increases sharply while validation error remains zero.

💡 Hint

Think about how more data helps the model learn better but eventually gains become smaller.

As training data increases, the model learns better, so both training and validation errors decrease. Eventually, they reach a plateau where adding more data doesn't improve performance much.

❓ Predict Output

intermediate

Output of learning curve plot code

What will be the output of the following Python code snippet using scikit-learn's learning_curve function?

ML Python

from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.model_selection import learning_curve
import numpy as np

X, y = load_iris(return_X_y=True)
train_sizes, train_scores, test_scores = learning_curve(SVC(kernel='linear'), X, y, train_sizes=np.linspace(0.1, 0.5, 5), cv=3)
print(train_sizes)
print(np.round(train_scores.mean(axis=1), 2))
print(np.round(test_scores.mean(axis=1), 2))

A

[15 30 45 60 75]
[0.8 0.85 0.9 0.95 1. ]
[0.7 0.75 0.8 0.85 0.9]

B

[15 30 45 60 75]
[0.5 0.5 0.5 0.5 0.5]
[0.5 0.5 0.5 0.5 0.5]

C

[10 20 30 40 50]
[1. 1. 1. 1. 1.]
[1. 1. 1. 1. 1.]

D

[15 30 45 60 75]
[1.   0.98 0.97 0.96 0.95]
[0.95 0.94 0.93 0.92 0.91]

💡 Hint

Check the size of the iris dataset and the train_sizes parameter.

The iris dataset has 150 samples. The train_sizes parameter creates 5 sizes from 10% to 50%, so sizes are [15,30,45,60,75]. The training scores are close to perfect for linear SVM, and test scores are slightly lower but close.

❓ Hyperparameter

advanced

Effect of model complexity on learning curves

How does increasing model complexity typically affect the shape of training and validation learning curves?

A Both training and validation errors increase with model complexity.

B Training error decreases but validation error first decreases then increases with complexity.

C Training error increases and validation error decreases as complexity grows.

D Training error decreases and validation error decreases steadily with complexity.

💡 Hint

Think about overfitting and underfitting effects.

As model complexity increases, training error usually decreases because the model fits training data better. Validation error decreases initially but then increases due to overfitting.

❓ Metrics

advanced

Interpreting learning curve metrics

If a learning curve shows a large gap between training and validation errors that does not decrease with more data, what does this indicate?

A The model is perfectly fitted with no issues.

B The model is underfitting and needs more features.

C The model is overfitting and more training data might help.

D The data is too noisy and cannot be learned.

💡 Hint

A large gap usually means the model fits training data too well but fails on new data.

A persistent large gap between training and validation errors suggests overfitting. Adding more data can help the model generalize better.

🔧 Debug

expert

Diagnosing incorrect learning curve results

A user runs learning_curve on a classification task but notices training scores are always lower than validation scores. What is the most likely cause?

A The scoring metric is incorrectly set to a metric that favors validation data.

B The model is too simple and underfits the training data.

C The training data is smaller than validation data causing unstable training scores.

D The cross-validation splits are not stratified, causing unbalanced classes in training.

💡 Hint

Think about how class distribution affects training and validation scores.

If cross-validation splits are not stratified, training folds may have fewer samples of some classes, causing poor training scores compared to validation.

Practice

1. What does a learning curve primarily show in machine learning?

easy

A. The hardware used for training the model

B. How model performance changes with more training data

C. The number of features in the dataset

D. The exact time a model takes to train

Solution

1. Step 1: Understand the purpose of learning curves

   Learning curves plot model performance (like accuracy or error) against the amount of training data.

2. Step 2: Identify what is shown on the curve

   The curve shows how well the model learns as it sees more data, helping to spot underfitting or overfitting.

3. Final Answer:

   How model performance changes with more training data -> Option B

4. Quick Check:

   Learning curve = model performance vs data size [OK]

Hint: Learning curves show performance vs data size [OK]

Common Mistakes:

• Confusing learning curves with training time plots
• Thinking learning curves show feature counts
• Assuming hardware affects learning curves directly

2. Which of the following is the correct way to plot a learning curve in Python using scikit-learn?

easy

A. import sklearn.plot as sp; sp.learning_curve(model, X, y)

B. from sklearn.learning import curve_plot; curve_plot(model, X, y)

C. from sklearn.model_selection import learning_curve; train_sizes, train_scores, test_scores = learning_curve(model, X, y)

D. from sklearn.model_selection import train_curve; train_curve(model, X, y)

Solution

1. Step 1: Recall the correct scikit-learn function for learning curves

   The function is learning_curve from sklearn.model_selection.

2. Step 2: Check the import and usage syntax

   from sklearn.model_selection import learning_curve; train_sizes, train_scores, test_scores = learning_curve(model, X, y) correctly imports learning_curve and calls it with model, features X, and labels y.

3. Final Answer:

   from sklearn.model_selection import learning_curve; train_sizes, train_scores, test_scores = learning_curve(model, X, y) -> Option C

4. Quick Check:

   Correct import and call = from sklearn.model_selection import learning_curve; train_sizes, train_scores, test_scores = learning_curve(model, X, y) [OK]

Hint: Use sklearn.model_selection.learning_curve to plot learning curves [OK]

Common Mistakes:

• Using wrong module or function names
• Trying to import non-existent functions
• Confusing learning_curve with train_test_split

3. Given this code snippet, what will be the shape of train_sizes if X has 100 samples?

from sklearn.model_selection import learning_curve
train_sizes, train_scores, test_scores = learning_curve(model, X, y, train_sizes=[0.1, 0.5, 1.0])

medium

A. (100,)

B. (1,)

C. (3, 100)

D. (3,)

Solution

1. Step 1: Understand train_sizes parameter

   train_sizes is given as a list with 3 values: 0.1, 0.5, and 1.0, representing fractions of the training data.

2. Step 2: Determine output shape of train_sizes

   The learning_curve function returns train_sizes as an array with length equal to the number of training sizes specified, so shape is (3,).

3. Final Answer:

   (3,) -> Option D

4. Quick Check:

   train_sizes length = number of fractions given [OK]

Hint: train_sizes shape matches number of training fractions [OK]

Common Mistakes:

• Assuming train_sizes matches number of samples
• Confusing train_sizes with train_scores shape
• Thinking train_sizes is always (100,)

4. You plotted a learning curve but see both training and validation errors are high and close. What is the likely issue?

medium

A. The model is underfitting the data

B. The model is overfitting the data

C. There is a data leakage problem

D. The training data is too large

Solution

1. Step 1: Analyze error levels on learning curve

   High training error means the model cannot fit the training data well.

2. Step 2: Compare training and validation errors

   Both errors being high and close indicates underfitting, where the model is too simple.

3. Final Answer:

   The model is underfitting the data -> Option A

4. Quick Check:

   High train & val error = underfitting [OK]

Hint: High train and val errors mean underfitting [OK]

Common Mistakes:

• Confusing underfitting with overfitting
• Assuming data leakage causes high errors
• Thinking large data size causes this pattern

5. You have a learning curve where training error is very low but validation error is high and does not improve with more data. What is the best next step?

hard

A. Try a simpler model to reduce overfitting

B. Collect more training data to improve validation

C. Increase model complexity to fit training data better

D. Stop training because the model is perfect

Solution

1. Step 1: Interpret the learning curve pattern

   Low training error but high validation error means the model fits training data too well but fails on new data, indicating overfitting.

2. Step 2: Decide the best action to fix overfitting

   Using a simpler model reduces complexity and helps generalize better, improving validation error.

3. Final Answer:

   Try a simpler model to reduce overfitting -> Option A

4. Quick Check:

   Low train error + high val error = overfitting; simplify model [OK]

Hint: Low train error + high val error means overfitting; simplify model [OK]

Common Mistakes:

• Collecting more data won't fix overfitting alone
• Increasing complexity worsens overfitting
• Assuming model is perfect with low training error