Cross-validation (K-fold) in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why use K-fold cross-validation?

Imagine you have a small dataset and want to estimate how well your machine learning model will perform on new data. Why is K-fold cross-validation a better choice than a single train-test split?

A It uses all data points for both training and testing, reducing bias in performance estimates.

B It trains the model multiple times on the same training set to improve accuracy.

C It splits data randomly once, which is faster and more reliable than multiple splits.

D It only tests the model on the largest portion of data to get the best score.

💡 Hint

Think about how using all data for testing at some point helps get a fair estimate.

K-fold cross-validation splits data into K parts and uses each part once as test data while training on the rest. This way, every data point is tested exactly once, giving a less biased and more reliable estimate of model performance than a single train-test split.

❓ Predict Output

intermediate

Output of K-fold split indices

What will be the output of the following Python code that uses KFold from scikit-learn?

ML Python

from sklearn.model_selection import KFold
import numpy as np

X = np.array([10, 20, 30, 40, 50])
kf = KFold(n_splits=2, shuffle=False)

splits = []
for train_index, test_index in kf.split(X):
    splits.append((train_index.tolist(), test_index.tolist()))

print(splits)

A [([3, 4], [0, 1, 2]), ([0, 1, 2], [3, 4])]

B [([0, 1, 2], [3, 4]), ([3, 4], [0, 1, 2])]

C [([1, 2, 3], [0, 4]), ([0, 4], [1, 2, 3])]

D [([2, 3, 4], [0, 1]), ([0, 1], [2, 3, 4])]

💡 Hint

Remember KFold splits data into equal parts without shuffling by default.

KFold with n_splits=2 splits the array into two parts: first 2 elements and last 3 elements. The first fold uses indices [0,1] as test and [2,3,4] as train, and the second fold reverses this.

❓ Model Choice

advanced

Choosing K for K-fold cross-validation

You have a dataset with 1000 samples. You want to use K-fold cross-validation to estimate model performance. Which choice of K balances bias and variance best?

A K = 5, because it provides a good balance between bias and variance for many datasets.

B K = 1000, because leave-one-out cross-validation always gives the best estimate.

C K = 1, because using the whole dataset for training and testing is most accurate.

D K = 2, because fewer folds reduce computation time and variance.

💡 Hint

Think about how increasing K affects bias and variance of the estimate.

Choosing K=5 or 10 is common because it balances bias and variance well. Too few folds (like 2) can give high bias, and too many folds (like leave-one-out) can have high variance and be computationally expensive.

❓ Metrics

advanced

Calculating average accuracy from K-fold results

You performed 4-fold cross-validation and got these accuracy scores for each fold: [0.82, 0.85, 0.80, 0.83]. What is the correct average accuracy to report?

A 0.8250

B 0.83

C 0.825

D 0.83 with standard deviation 0.02

💡 Hint

Calculate the mean of the scores.

The average accuracy is the sum of all accuracies divided by the number of folds: (0.82 + 0.85 + 0.80 + 0.83) / 4 = 0.825.

🔧 Debug

expert

Why does this K-fold code raise an error?

Consider this Python code snippet using KFold. It raises an error. What is the cause?

ML Python

from sklearn.model_selection import KFold
import numpy as np

X = np.array([1, 2, 3])
kf = KFold(n_splits=5)

for train_index, test_index in kf.split(X):
    print('Train:', train_index, 'Test:', test_index)

A IndexError because test indices exceed array length.

B ValueError because n_splits (5) cannot be greater than the number of samples (3).

C TypeError because KFold expects a list, not a numpy array.

D RuntimeError because KFold requires shuffle=true for small datasets.

💡 Hint

Check the relationship between n_splits and dataset size.

KFold requires that n_splits be less than or equal to the number of samples. Here, n_splits=5 but only 3 samples exist, causing a ValueError.

Practice

1. What is the main purpose of K-fold cross-validation in machine learning?

easy

A. To evaluate model performance on different subsets of data

B. To increase the size of the training dataset

C. To reduce the number of features in the dataset

D. To speed up the training process

Solution

1. Step 1: Understand K-fold cross-validation concept

   K-fold cross-validation splits the data into K parts and trains the model K times, each time using a different part as test data.

2. Step 2: Identify the main goal

   This process helps evaluate how well the model performs on unseen data by testing on different subsets.

3. Final Answer:

   To evaluate model performance on different subsets of data -> Option A

4. Quick Check:

   Cross-validation = model evaluation [OK]

Hint: K-fold tests model on all data parts for reliable results [OK]

Common Mistakes:

• Thinking it increases training data size
• Confusing it with feature selection
• Assuming it speeds up training

2. Which of the following is the correct way to import KFold from scikit-learn in Python?

easy

A. from sklearn.cross_validation import KFold

B. import KFold from sklearn.cross_validation

C. from sklearn.model_selection import KFold

D. import KFold from sklearn.model_selection

Solution

1. Step 1: Recall the current scikit-learn module for KFold

   KFold is part of the model_selection module in scikit-learn.

2. Step 2: Check correct import syntax

   The correct syntax is from sklearn.model_selection import KFold. Other options use outdated or incorrect modules or syntax.

3. Final Answer:

   from sklearn.model_selection import KFold -> Option C

4. Quick Check:

   Correct import = from sklearn.model_selection import KFold [OK]

Hint: Use model_selection module for KFold import [OK]

Common Mistakes:

• Using deprecated sklearn.cross_validation module
• Wrong import syntax like 'import KFold from ...'
• Confusing module names

3. What will be the output of the following Python code snippet?

from sklearn.model_selection import KFold
import numpy as np

X = np.array([10, 20, 30, 40, 50])
kf = KFold(n_splits=5)
for train_index, test_index in kf.split(X):
    print(test_index)

medium

A. [0 1 2 3]
[1 2 3 4]
[0 2 3 4]
[0 1 3 4]
[0 1 2 4]

B. [0 1]
[2 3]
[4]

C. [4]
[3]
[2]
[1]
[0]

D. [0]
[1]
[2]
[3]
[4]

Solution

1. Step 1: Understand KFold with n_splits=5 on 5 elements

   With 5 splits, each fold uses 4 elements for training and 1 element for testing.

2. Step 2: Check test indices printed in each fold

   Each fold's test index is a single element index from 0 to 4, printed one per line.

3. Final Answer:

   [0] [1] [2] [3] [4] -> Option D

4. Quick Check:

   Each fold tests one unique index [OK]

Hint: With 5 splits, test sets have 1 unique index each [OK]

Common Mistakes:

• Expecting multiple indices per test set
• Confusing train and test indices
• Misunderstanding KFold split behavior

4. Identify the error in this KFold usage code snippet:

from sklearn.model_selection import KFold
kf = KFold(n_splits=5)
X = [1, 2, 3, 4]
for train_idx, test_idx in kf.split(X):
    print(train_idx, test_idx)

medium

A. n_splits cannot be greater than the number of samples

B. X must be a numpy array, not a list

C. train_idx and test_idx should be swapped in the loop

D. Missing shuffle=True in KFold initialization

Solution

1. Step 1: Check n_splits vs data size

   n_splits=5 but X has only 4 samples (5 > 4).

2. Step 2: Understand KFold requirement

   KFold requires n_splits to be less than or equal to the number of samples, otherwise it raises a ValueError.

3. Final Answer:

   n_splits cannot be greater than the number of samples -> Option A

4. Quick Check:

   n_splits ≤ samples [OK]

Hint: n_splits must not exceed sample count [OK]

Common Mistakes:

• Using n_splits larger than dataset size
• Thinking list input causes error
• Swapping train and test indices

5. You want to use 10-fold cross-validation on a dataset with 100 samples. However, your model training is very slow. Which strategy can you use to still evaluate your model reliably but reduce training time?

hard

A. Use only one fold for testing and ignore others

B. Use 5-fold cross-validation instead of 10-fold

C. Use 10-fold cross-validation but shuffle=False

D. Use the entire dataset as training without validation

Solution

1. Step 1: Understand the trade-off between folds and training time

   More folds mean more training rounds, increasing time. Reducing folds reduces training rounds.

2. Step 2: Evaluate options for reducing training time

   Using 5-fold halves the number of training rounds compared to 10-fold, still providing reliable evaluation.

3. Step 3: Check other options

   Using entire data without validation loses evaluation, shuffle=False doesn't reduce folds, ignoring folds loses reliability.

4. Final Answer:

   Use 5-fold cross-validation instead of 10-fold -> Option B

5. Quick Check:

   Fewer folds = faster training [OK]

Hint: Reduce folds to speed up cross-validation [OK]

Common Mistakes:

• Skipping validation entirely
• Ignoring folds reduces reliability
• Thinking shuffle affects speed