K-fold cross-validation in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of K-fold split indices

Given the following code snippet using sklearn's KFold, what will be the output of the printed train and test indices for fold 1?

TensorFlow

from sklearn.model_selection import KFold
import numpy as np

X = np.array([10, 20, 30, 40, 50])
kf = KFold(n_splits=5, shuffle=False)

for fold, (train_index, test_index) in enumerate(kf.split(X), 1):
    if fold == 1:
        print('Train indices:', train_index)
        print('Test indices:', test_index)

A

Train indices: [1 2 3 4]
Test indices: [0]

B

Train indices: [0 1 2 3]
Test indices: [4]

C

Train indices: [0 2 3 4]
Test indices: [1]

D

Train indices: [0 1 3 4]
Test indices: [2]

💡 Hint

Remember that KFold splits data into equal parts without shuffling by default.

With 5 splits and no shuffle, fold 1 test set is the first element (index 0), so train indices are the rest.

❓ Model Choice

intermediate

Choosing model evaluation strategy

You have a small dataset and want to estimate your TensorFlow model's performance reliably. Which cross-validation strategy is best to reduce variance in performance estimates?

A Use 10-fold cross-validation with shuffling

B Use a single train-test split with 80% training data

C Use 5-fold cross-validation without shuffling

D Use leave-one-out cross-validation without shuffling

💡 Hint

More folds with shuffling usually give better performance estimates on small datasets.

10-fold CV with shuffling reduces bias and variance better than fewer folds or no shuffling.

❓ Hyperparameter

advanced

Effect of number of folds on training time

If you increase the number of folds in K-fold cross-validation from 5 to 10 for training a TensorFlow model, what is the expected effect on total training time?

A Training time will be unpredictable and may increase or decrease randomly

B Training time will roughly double because the model trains on more folds

C Training time will stay the same because total data trained is unchanged

D Training time will decrease because each fold trains on less data

💡 Hint

Each fold requires training a separate model on a subset of data.

Doubling folds means training twice as many models, roughly doubling total training time.

❓ Metrics

advanced

Calculating average accuracy from K-fold results

You performed 4-fold cross-validation on a classification model and got accuracies: [0.82, 0.85, 0.80, 0.83]. What is the correct average accuracy to report?

A 0.825

B 0.8250

C 82.5%

D All of the above

💡 Hint

Average accuracy is the mean of fold accuracies; formatting can vary.

All options represent the same average accuracy in different formats.

🔧 Debug

expert

Identifying error in K-fold TensorFlow training loop

Consider this TensorFlow K-fold training loop code snippet. What error will it raise? ```python from sklearn.model_selection import KFold import tensorflow as tf import numpy as np X = np.random.rand(100, 10) y = np.random.randint(0, 2, 100) kf = KFold(n_splits=5, shuffle=True) for train_idx, test_idx in kf.split(X): model = tf.keras.Sequential([ tf.keras.layers.Dense(1, activation='sigmoid') ]) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) model.fit(X[train_idx], y[train_idx], epochs=3) loss, acc = model.evaluate(X[test_idx], y[test_idx]) print(f"Test accuracy: {acc}") print(model.predict(X)) ```

A No error; code runs and prints predictions

B ValueError due to shape mismatch in model.predict input

C NameError because model is undefined outside the loop

D RuntimeError because model.fit is called multiple times on same model

💡 Hint

Check where the variable 'model' is defined and used.

The variable 'model' is defined inside the loop and not accessible after the loop ends, causing NameError.

Practice

1. What is the main purpose of using K-fold cross-validation in TensorFlow model training?

easy

A. To increase the size of the training dataset

B. To evaluate model performance on different parts of the data

C. To speed up the training process

D. To reduce the number of model parameters

Solution

1. Step 1: Understand K-fold cross-validation concept

   K-fold cross-validation splits the dataset into K parts and trains the model K times, each time using a different part as test data.

2. Step 2: Identify the purpose of this process

   This method helps evaluate how well the model performs on unseen data by testing it on different subsets.

3. Final Answer:

   To evaluate model performance on different parts of the data -> Option B

4. Quick Check:

   Purpose = Evaluate model on different data parts [OK]

Hint: K-fold tests model on all data parts for better evaluation [OK]

Common Mistakes:

• Thinking it increases training data size
• Believing it speeds up training
• Confusing it with parameter tuning

2. Which of the following is the correct way to split data into 5 folds using TensorFlow's tf.data API for K-fold cross-validation?

easy

A. Use tf.data.Dataset.shard(num_shards=5, index=i) inside a loop over i from 0 to 4

B. Use tf.data.Dataset.batch(5) to create 5 folds

C. Use tf.data.Dataset.repeat(5) to repeat data 5 times

D. Use tf.data.Dataset.shuffle(5) to shuffle into 5 folds

Solution

1. Step 1: Recall TensorFlow data splitting method

   TensorFlow's shard method splits a dataset into equal parts by specifying total shards and the index of the shard.

2. Step 2: Match method to K-fold splitting

   For 5 folds, use num_shards=5 and loop over index from 0 to 4 to get each fold.

3. Final Answer:

   Use tf.data.Dataset.shard(num_shards=5, index=i) inside a loop over i from 0 to 4 -> Option A

4. Quick Check:

   Sharding splits data into folds correctly [OK]

Hint: Use shard with num_shards = K for K-fold splits [OK]

Common Mistakes:

• Using batch instead of shard for splitting
• Repeating data instead of splitting
• Shuffling does not create folds

3. Consider this code snippet for 3-fold cross-validation in TensorFlow:

for i in range(3):
    test_data = dataset.shard(num_shards=3, index=i)
    train_data = dataset.shard(num_shards=3, index=(i+1) % 3)
    # model training code here
    print(f'Fold {i} train size:', len(list(train_data)))
    print(f'Fold {i} test size:', len(list(test_data)))

What will be the output sizes for train and test data in each fold if the original dataset has 9 samples?

medium

A. Train size: 9, Test size: 0 for all folds

B. Train size: 6, Test size: 3 for all folds

C. Train size: 3, Test size: 3 for all folds

D. Train size: 3, Test size: 6 for all folds

Solution

1. Step 1: Understand shard usage in code

   Each fold uses dataset.shard(num_shards=3, index=i) for test data and dataset.shard(num_shards=3, index=(i+1) % 3) for train data. This means train and test data are both single shards, not complementary sets.

2. Step 2: Calculate sizes based on shards

   Each shard has 3 samples (9/3). So test_data has 3 samples. But train_data is only one shard (3 samples), not the other two shards combined (6 samples). So train and test overlap and total is 6 samples, missing 3 samples.

3. Step 3: Check code behavior

   Since train_data and test_data are both shards, the rest of the data is ignored. The code prints sizes of 3 for train and 3 for test, but the rest 3 samples are missing from both.

4. Final Answer:

   Train size: 3, Test size: 3 for all folds -> Option C

5. Quick Check:

   Shard selects one part only, so train = 3, test = 3 [OK]

Hint: Shard picks one part; train must combine other shards [OK]

Common Mistakes:

• Assuming train is all except test shard
• Counting total dataset size for train
• Ignoring shard overlap

4. You wrote this code for 4-fold cross-validation but get an error:

for i in range(4):
    test_data = dataset.shard(num_shards=4, index=i)
    train_data = dataset.shard(num_shards=4, index!=i)
    # train model here

What is the error and how to fix it?

medium

A. Syntax error in index!=i; use a loop to combine other shards

B. Wrong number of shards; should be 5 instead of 4

C. Missing shuffle before sharding causes error

D. Using shard twice causes conflict; use split instead

Solution

1. Step 1: Identify syntax error in shard index

   The expression index!=i is invalid syntax for the index parameter, which expects an integer, not a boolean expression.

2. Step 2: Correct approach to get train data

   To get train data (all shards except test shard), you must combine shards with indices other than i using a loop or filtering, not a single shard call with index!=i.

3. Final Answer:

   Syntax error in index!=i; use a loop to combine other shards -> Option A

4. Quick Check:

   Shard index must be integer, not condition [OK]

Hint: Shard index needs integer; combine others in loop [OK]

Common Mistakes:

• Using boolean condition as shard index
• Assuming shard auto-excludes test fold
• Not combining shards for train data

5. You want to implement 5-fold cross-validation in TensorFlow to train a neural network on a dataset with 1000 samples. Which approach correctly creates train and test datasets for each fold to avoid data leakage?

hard

A. For each fold i, use test_data = dataset.shard(5, i) and train_data = dataset.filter(lambda x: x not in test_data)

B. For each fold i, use test_data = dataset.shard(5, i) and train_data = dataset.shard(5, (i+1) % 5)

C. For each fold i, use dataset.shuffle(1000) and split 80% train, 20% test randomly

D. For each fold i, use test_data = dataset.shard(5, i) and combine all other shards except i for train_data

Solution

1. Step 1: Understand correct train-test split in K-fold

   Test data is one shard. Train data must be all other shards combined to avoid overlap and data leakage.

2. Step 2: Evaluate options

   For each fold i, use test_data = dataset.shard(5, i) and train_data = dataset.filter(lambda x: x not in test_data) tries to filter dataset by membership, which is not supported directly in TensorFlow datasets. For each fold i, use test_data = dataset.shard(5, i) and train_data = dataset.shard(5, (i+1) % 5) uses only one shard for train, missing other data. For each fold i, use dataset.shuffle(1000) and split 80% train, 20% test randomly shuffles and splits randomly, not K-fold. For each fold i, use test_data = dataset.shard(5, i) and combine all other shards except i for train_data correctly combines all shards except the test shard for training.

3. Final Answer:

   For each fold i, use test_data = dataset.shard(5, i) and combine all other shards except i for train_data -> Option D

4. Quick Check:

   Train = all shards except test shard [OK]

Hint: Train = all shards except test shard combined [OK]

Common Mistakes:

• Using only one shard for train data
• Filtering datasets by membership incorrectly
• Random split instead of K-fold