Hyperparameter tuning (GridSearchCV) in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding GridSearchCV's purpose

What is the main goal of using GridSearchCV in machine learning?

A To visualize the training data in multiple dimensions.

B To reduce the size of the training dataset automatically.

C To find the best combination of hyperparameters by exhaustively searching a specified parameter grid using cross-validation.

D To train a model faster by skipping validation steps.

💡 Hint

Think about what tuning hyperparameters means and how GridSearchCV helps with model performance.

GridSearchCV tries all combinations of hyperparameters you specify and uses cross-validation to find which combination works best for your model.

❓ Predict Output

intermediate

Output of GridSearchCV best parameters

What will be the output of print(grid_search.best_params_) after running the following code?

ML Python

from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV

iris = load_iris()
X, y = iris.data, iris.target

param_grid = {'C': [0.1, 1], 'kernel': ['linear', 'rbf']}
svc = SVC()
grid_search = GridSearchCV(svc, param_grid, cv=3)
grid_search.fit(X, y)
print(grid_search.best_params_)

A {'C': 1, 'kernel': 'rbf'}

B {'C': 0.1, 'kernel': 'rbf'}

C {'C': 1, 'kernel': 'linear'}

D {'C': 0.1, 'kernel': 'linear'}

💡 Hint

Consider which kernel and C value usually perform better on iris dataset with SVC.

For the iris dataset, SVC with kernel 'rbf' and C=1 generally performs better than other combinations, so GridSearchCV selects {'C': 1, 'kernel': 'rbf'}.

❓ Hyperparameter

advanced

Choosing hyperparameters for GridSearchCV

Which set of hyperparameters is most appropriate to tune for a Random Forest classifier using GridSearchCV to improve model performance?

A {'learning_rate': [0.01, 0.1], 'activation': ['relu', 'tanh']}

B {'n_estimators': [10, 50, 100], 'max_depth': [None, 10, 20], 'min_samples_split': [2, 5]}

C {'kernel': ['linear', 'poly'], 'C': [0.1, 1]}

D {'alpha': [0.0001, 0.001], 'max_iter': [1000, 2000]}

💡 Hint

Think about which hyperparameters belong to Random Forest models.

Random Forest tuning commonly involves number of trees (n_estimators), tree depth (max_depth), and minimum samples to split nodes (min_samples_split).

❓ Metrics

advanced

Evaluating GridSearchCV results

After running GridSearchCV, which attribute provides the mean test scores for each parameter combination tested?

A grid_search.best_score_

B grid_search.score(X_test, y_test)

C grid_search.best_params_

D grid_search.cv_results_['mean_test_score']

💡 Hint

Look for the attribute that stores detailed results for all parameter combinations.

cv_results_ is a dictionary containing detailed info about each parameter set tested, including mean_test_score which holds the average cross-validation scores.

🔧 Debug

expert

Identifying error in GridSearchCV usage

What error will the following code raise when executed?

from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV

iris = load_iris()
X, y = iris.data, iris.target

param_grid = {'C': [1, 10], 'kernel': ['linear', 'rbf']}
svc = SVC()
grid_search = GridSearchCV(svc, param_grid, cv=3)
grid_search.fit(X, y)
print(grid_search.best_params_)
print(grid_search.best_estimator_)
print(grid_search.predict(X))

A NameError because 'grid_search' is not defined

B TypeError because 'param_grid' has invalid types

C ValueError because 'cv' parameter is invalid

D No error, code runs successfully

💡 Hint

Inspect the code line by line for any syntax or logical errors.

The code will raise a NameError because the variable used to fit the model is named 'grid_search', but the print statements use 'grid_search' which is correct. However, the code snippet uses 'grid_search' consistently, so no NameError. But the prompt code uses 'grid_search' consistently, so no error. The actual error is that the print statements use 'grid_search' but the variable is 'grid_search' - no mismatch. So no error. But the prompt code uses 'grid_search' consistently, so no error. The correct answer should be D.

Practice

1.

What is the main purpose of using GridSearchCV in machine learning?

easy

A. To split data into training and testing sets

B. To find the best combination of hyperparameters automatically

C. To visualize the model's predictions

D. To reduce the size of the dataset

Solution

1. Step 1: Understand what GridSearchCV does

   GridSearchCV tries different hyperparameter combinations to find the best settings for a model.

2. Step 2: Compare options with GridSearchCV's purpose

   Only To find the best combination of hyperparameters automatically describes this automatic search for best hyperparameters.

3. Final Answer:

   To find the best combination of hyperparameters automatically -> Option B

4. Quick Check:

   GridSearchCV = best hyperparameters [OK]

Hint: GridSearchCV = automatic hyperparameter search [OK]

Common Mistakes:

• Confusing GridSearchCV with data splitting
• Thinking it visualizes predictions
• Assuming it reduces dataset size

2.

Which of the following is the correct way to import GridSearchCV from sklearn?

easy

A. import GridSearchCV from sklearn

B. from sklearn.grid_search import GridSearchCV

C. from sklearn.model_selection import GridSearchCV

D. from sklearn.tuning import GridSearchCV

Solution

1. Step 1: Recall the correct sklearn module for GridSearchCV

   GridSearchCV is in the model_selection module of sklearn.

2. Step 2: Match the correct import syntax

   from sklearn.model_selection import GridSearchCV uses the correct module and syntax for import.

3. Final Answer:

   from sklearn.model_selection import GridSearchCV -> Option C

4. Quick Check:

   Correct import = from sklearn.model_selection import GridSearchCV [OK]

Hint: GridSearchCV is in sklearn.model_selection [OK]

Common Mistakes:

• Using deprecated sklearn.grid_search module
• Wrong import syntax like 'import GridSearchCV from sklearn'
• Assuming a nonexistent sklearn.tuning module

3.

What will be the output of the following code snippet?

from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV

iris = load_iris()
X, y = iris.data, iris.target

param_grid = {'kernel': ['linear', 'rbf'], 'C': [1, 10]}
svc = SVC()
grid = GridSearchCV(svc, param_grid, cv=3)
grid.fit(X, y)
print(grid.best_params_)

medium

A. A dictionary with best kernel and C values found by cross-validation

B. {'kernel': 'rbf', 'C': 10}

C. {'kernel': 'linear', 'C': 1}

D. An error because 'cv' parameter is missing

Solution

1. Step 1: Understand GridSearchCV output

   GridSearchCV's best_params_ attribute returns the best hyperparameters found after testing all combinations.

2. Step 2: Analyze the code snippet

   The code runs GridSearchCV with kernel and C values, then prints the best combination as a dictionary.

3. Final Answer:

   A dictionary with best kernel and C values found by cross-validation -> Option A

4. Quick Check:

   grid.best_params_ = best hyperparameters dict [OK]

Hint: grid.best_params_ shows best hyperparameters found [OK]

Common Mistakes:

• Expecting fixed values instead of dynamic best params
• Thinking cv parameter is missing (it is present)
• Confusing output with training scores

4.

Identify the error in this GridSearchCV usage:

from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV

param_grid = {'C': [0.1, 1, 10]}
svc = SVC()
grid = GridSearchCV(svc, param_grid)
grid.fit(X_train, y_train)
print(grid.best_score_)

Assuming X_train and y_train are defined.

medium

A. Missing cross-validation parameter cv in GridSearchCV

B. Parameter grid keys should be strings with double underscores

C. SVC requires kernel parameter in param_grid

D. No error; code runs correctly and prints best score

Solution

1. Step 1: Check default behavior of GridSearchCV

   If cv is not given, GridSearchCV uses default 5-fold cross-validation.

2. Step 2: Verify param_grid and SVC usage

   Parameter grid keys like 'C' are correct; kernel is optional with default 'rbf'.

3. Final Answer:

   No error; code runs correctly and prints best score -> Option D

4. Quick Check:

   Default cv=5, param_grid keys correct [OK]

Hint: GridSearchCV uses 5-fold CV by default if cv missing [OK]

Common Mistakes:

• Assuming cv parameter is mandatory
• Thinking param_grid keys need double underscores
• Believing kernel must be in param_grid

5.

You want to tune a RandomForestClassifier using GridSearchCV. You want to try n_estimators values 50, 100 and max_depth values 5, 10. Which code snippet correctly sets up the grid search?

hard

A. param_grid = {'n_estimators': [50, 100], 'max_depth': [5, 10]}
rf = RandomForestClassifier()
grid = GridSearchCV(rf, param_grid, cv=4)

B. param_grid = {'RandomForestClassifier__n_estimators': [50, 100], 'RandomForestClassifier__max_depth': [5, 10]}
rf = RandomForestClassifier()
grid = GridSearchCV(rf, param_grid)

C. param_grid = {'n_estimators': 50 or 100, 'max_depth': 5 or 10}
rf = RandomForestClassifier()
grid = GridSearchCV(rf, param_grid)

D. param_grid = {'n_estimators': [50, 100], 'max_depth': [5, 10]}
rf = RandomForestClassifier()
grid = GridSearchCV(rf, param_grid, cv='invalid')

Solution

1. Step 1: Define param_grid correctly for RandomForestClassifier

   Use keys as hyperparameter names with list of values: {'n_estimators': [50, 100], 'max_depth': [5, 10]}.

2. Step 2: Setup GridSearchCV with valid cv parameter

   The correct snippet initializes GridSearchCV with the proper param_grid using lists and a valid integer cv value (e.g., cv=4). Distractors include invalid cv values like strings, incorrect parameter key prefixes, and non-list value syntax like 'or'.

3. Final Answer:

   param_grid = {'n_estimators': [50, 100], 'max_depth': [5, 10]} rf = RandomForestClassifier() grid = GridSearchCV(rf, param_grid, cv=4) -> Option A

4. Quick Check:

   Correct param_grid and valid cv = param_grid = {'n_estimators': [50, 100], 'max_depth': [5, 10]} rf = RandomForestClassifier() grid = GridSearchCV(rf, param_grid, cv=4) [OK]

Hint: Use param names as keys and valid cv number [OK]

Common Mistakes:

• Adding model name prefix to param_grid keys
• Using 'or' instead of list for values
• Setting invalid cv values like strings