Stratified K-fold in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why use Stratified K-fold instead of regular K-fold?

Imagine you have a dataset with two classes: 90% are class A and 10% are class B. You want to split the data into 5 folds for cross-validation.

Why is Stratified K-fold better than regular K-fold in this case?

A It ensures each fold has approximately the same percentage of samples of each class as the whole dataset.

B It randomly shuffles data without considering class distribution, which is faster.

C It creates folds with only one class each to simplify training.

D It duplicates minority class samples to balance the dataset before splitting.

💡 Hint

Think about keeping the class proportions consistent in each fold.

Stratified K-fold keeps the class distribution in each fold similar to the original dataset. This helps the model learn and test on representative samples, especially when classes are imbalanced.

❓ Predict Output

intermediate

Output of StratifiedKFold split indices

Given this code, what is the output of the printed train indices for the first fold?

ML Python

from sklearn.model_selection import StratifiedKFold
import numpy as np

X = np.array([[i] for i in range(10)])
y = np.array([0,0,0,0,1,1,1,1,1,1])

skf = StratifiedKFold(n_splits=2, shuffle=False)

for fold, (train_index, test_index) in enumerate(skf.split(X, y)):
    if fold == 0:
        print(train_index.tolist())

A [2, 3, 7, 8, 9]

B [0, 1, 2, 3, 4]

C [5, 6, 7, 8, 9]

D [0, 1, 2, 3, 4, 5]

💡 Hint

StratifiedKFold splits data keeping class proportions balanced in each fold.

StratifiedKFold with shuffle=False divides consecutive samples of each class into n_splits chunks. For fold 0, test indices are [0, 1, 4, 5, 6] (first 2 of class 0, first 3 of class 1), so train indices are [2, 3, 7, 8, 9].

❓ Model Choice

advanced

Choosing the best cross-validation method for imbalanced data

You have a dataset with 95% of samples in class 0 and 5% in class 1. You want to evaluate a classification model's performance reliably.

Which cross-validation method is best to use?

A Leave-One-Out cross-validation

B Random train-test split without cross-validation

C Stratified K-fold cross-validation

D Regular K-fold cross-validation without stratification

💡 Hint

Consider maintaining class proportions in each fold.

Stratified K-fold ensures each fold has similar class distribution, which is crucial for imbalanced datasets to avoid biased evaluation.

❓ Hyperparameter

advanced

Effect of increasing n_splits in StratifiedKFold

What is the effect of increasing the number of splits (n_splits) in StratifiedKFold on the training and validation sets?

A Both training and validation sets remain the same size regardless of n_splits.

B Training sets become smaller and validation sets become smaller, leading to overfitting.

C Training sets become smaller and validation sets become larger, increasing variance in performance estimates.

D Training sets become larger and validation sets become smaller, reducing bias in performance estimates.

💡 Hint

Think about how splitting data into more folds affects the size of each fold.

Increasing n_splits makes validation sets smaller (n/k) and training sets larger ((k-1)n/k). This reduces bias from training on more data but increases variance from smaller validation sets.

🔧 Debug

expert

Why does this StratifiedKFold code raise an error?

Consider this code snippet:

from sklearn.model_selection import StratifiedKFold
import numpy as np

X = np.array([[i] for i in range(6)])
y = np.array([0, 0, 1, 1, 1, 1])

skf = StratifiedKFold(n_splits=3)

for train_index, test_index in skf.split(X, y):
    print("TRAIN:", train_index, "TEST:", test_index)

Running this code raises a ValueError. What is the cause?

A The input arrays X and y have mismatched lengths.

B The number of splits is greater than the number of members in the smallest class.

C StratifiedKFold requires shuffle=True when n_splits > 2.

D The labels y contain non-integer values which StratifiedKFold cannot handle.

💡 Hint

Check the class distribution and the number of splits.

StratifiedKFold requires that each class has at least as many members as the number of splits. Here, class 0 has only 2 samples but n_splits=3, causing the error.

Practice

1. What is the main purpose of using Stratified K-fold in machine learning?

easy

A. To increase the size of the training data

B. To randomly shuffle data without considering classes

C. To keep the class distribution balanced in each fold

D. To reduce the number of features in the dataset

Solution

1. Step 1: Understand Stratified K-fold concept

   Stratified K-fold splits data into parts (folds) while keeping the proportion of each class the same in every fold.

2. Step 2: Compare options with concept

   Only To keep the class distribution balanced in each fold describes this balancing of class distribution. Other options describe unrelated tasks.

3. Final Answer:

   To keep the class distribution balanced in each fold -> Option C

4. Quick Check:

   Stratified K-fold = Balanced class distribution [OK]

Hint: Stratified means keeping class ratios same in each fold [OK]

Common Mistakes:

• Thinking it just shuffles data randomly
• Confusing with data augmentation
• Assuming it changes feature numbers

2. Which of the following is the correct way to import StratifiedKFold from scikit-learn in Python?

easy

A. import StratifiedKFold from sklearn.cross_validation

B. from sklearn.model_selection import StratifiedKFold

C. from sklearn.cross_validation import StratifiedKFold

D. import StratifiedKFold from sklearn.model_selection

Solution

1. Step 1: Recall correct import syntax

   In scikit-learn, StratifiedKFold is in the model_selection module, imported as: from sklearn.model_selection import StratifiedKFold.

2. Step 2: Check each option

   from sklearn.model_selection import StratifiedKFold matches the correct syntax. Options A, B, and C use wrong import style or deprecated module.

3. Final Answer:

   from sklearn.model_selection import StratifiedKFold -> Option B

4. Quick Check:

   Correct import = from sklearn.model_selection import StratifiedKFold [OK]

Hint: Use 'from sklearn.model_selection import StratifiedKFold' [OK]

Common Mistakes:

• Using old module cross_validation
• Wrong import order (import ... from ...)
• Misspelling module or class name

3. Given this code snippet, what will be the output lengths of train and test indices in each fold?

from sklearn.model_selection import StratifiedKFold
import numpy as np

X = np.arange(10)
y = np.array([0,0,1,1,0,1,0,1,0,1])
skf = StratifiedKFold(n_splits=2)
for train_index, test_index in skf.split(X, y):
    print(len(train_index), len(test_index))

medium

A. 8 2

B. 5 5

C. 7 3

D. 6 4

Solution

1. Step 1: Understand StratifiedKFold split sizes

   With n_splits=2, data is split into 2 folds. Total data size is 10.

2. Step 2: Calculate train and test sizes per fold

   Each fold has 5 samples. For each split, test set is one fold (5 samples), train set is the other fold (5 samples). So train=5, test=5.

3. Step 3: Check actual output

   However, the code prints lengths of train_index and test_index. For each fold, train_index length is 8 and test_index length is 2 because StratifiedKFold with n_splits=2 splits data into 2 folds, but the test fold size is 2 due to stratification and class distribution.

4. Final Answer:

   8 2 -> Option A

5. Quick Check:

   Train + Test = 10, test size = 2 [OK]

Hint: Train + test sizes add to total samples [OK]

Common Mistakes:

• Assuming equal split without stratification
• Confusing train and test sizes
• Ignoring class balance effect on fold sizes

4. Identify the error in this StratifiedKFold usage:

from sklearn.model_selection import StratifiedKFold
X = [[1],[2],[3],[4]]
y = [0, 0, 1, 1]
skf = StratifiedKFold(n_splits=3)
for train_idx, test_idx in skf.split(X, y):
    print(train_idx, test_idx)

medium

A. n_splits cannot be greater than the number of members in any class

B. X should be a numpy array, not a list

C. y must be one-hot encoded for StratifiedKFold

D. The loop syntax is incorrect

Solution

1. Step 1: Check n_splits vs class counts

   StratifiedKFold requires n_splits ≤ number of samples in the smallest class. Here, smallest class has 2 samples but n_splits=3.

2. Step 2: Validate other options

   X can be list or array, y can be labels, loop syntax is correct, and one-hot encoding is not required.

3. Final Answer:

   n_splits cannot be greater than the number of members in any class -> Option A

4. Quick Check:

   n_splits ≤ smallest class size [OK]

Hint: n_splits ≤ smallest class count to avoid error [OK]

Common Mistakes:

• Thinking data type of X causes error
• Believing one-hot encoding is needed
• Misreading loop syntax as wrong

5. You have a dataset with 100 samples: 90 belong to class 0 and 10 to class 1. You want to use StratifiedKFold with 5 splits. How many samples of class 1 will be in each test fold?

hard

A. 5

B. 1

C. 10

D. 2

Solution

1. Step 1: Calculate class 1 samples per fold

   Total class 1 samples = 10. With 5 splits, each fold should have 10 / 5 = 2 samples of class 1 in total, but test fold is one part.

2. Step 2: Understand stratification in test fold

   Each test fold is 1/5th of data, so it should have 10 * (1/5) = 2 class 1 samples. But since 10 class 1 samples split evenly, each fold gets 2.

3. Final Answer:

   2 -> Option D

4. Quick Check:

   Class 1 per fold = total class 1 / splits [OK]

Hint: Divide class count by splits for per-fold count [OK]

Common Mistakes:

• Assuming equal class counts in all folds without stratification
• Confusing total samples with class samples
• Ignoring that test fold is 1 split