Pipeline with GridSearchCV in ML Python

Introduction

We use a pipeline with GridSearchCV to try many settings for a model and preprocessing steps all at once. This helps find the best way to prepare data and train the model without mistakes.

When you want to test different data cleaning or scaling methods together with model settings.

When you want to avoid repeating code for preprocessing before training.

When you want to find the best model settings automatically by trying many options.

When you want to keep your code clean and easy to understand.

When you want to make sure your model works well on new data by tuning it carefully.

Syntax

ML Python

from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV

pipeline = Pipeline([
    ('step_name', transformer_or_model),
    ('model', estimator)
])

param_grid = {
    'step_name__parameter': [values],
    'model__parameter': [values]
}

grid_search = GridSearchCV(pipeline, param_grid, cv=number_of_folds)
grid_search.fit(X_train, y_train)

Use double underscores __ to set parameters for steps inside the pipeline.

cv means how many parts to split data for testing during tuning.

Examples

This example tries scaling with or without centering and different regularization strengths for logistic regression.

ML Python

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LogisticRegression())
])

param_grid = {
    'scaler__with_mean': [True, False],
    'clf__C': [0.1, 1, 10]
}

This example tries different numbers of PCA components and SVM kernels.

ML Python

pipeline = Pipeline([
    ('pca', PCA()),
    ('svc', SVC())
])

param_grid = {
    'pca__n_components': [2, 3, 4],
    'svc__kernel': ['linear', 'rbf']
}

Sample Model

This program loads the iris flower data, splits it, and creates a pipeline that scales data and trains an SVM model. It tries different scaling options and SVM settings to find the best combination. Finally, it prints the best settings and how well the model works on test data.

ML Python

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

# Load data
X, y = load_iris(return_X_y=True)

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create pipeline
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('svc', SVC())
])

# Parameter grid
param_grid = {
    'scaler__with_mean': [True, False],
    'svc__C': [0.1, 1, 10],
    'svc__kernel': ['linear', 'rbf']
}

# Grid search
grid_search = GridSearchCV(pipeline, param_grid, cv=3)
grid_search.fit(X_train, y_train)

# Best parameters
print('Best parameters:', grid_search.best_params_)

# Test accuracy
test_score = grid_search.score(X_test, y_test)
print(f'Test accuracy: {test_score:.2f}')

Important Notes

Always use pipelines to avoid data leakage during cross-validation.

GridSearchCV tries all combinations, so keep parameter lists small to save time.

You can add more preprocessing steps before the model in the pipeline.

Summary

Pipelines combine data steps and models into one object.

GridSearchCV finds the best settings by testing many options.

Use double underscores to set parameters inside pipeline steps.

Practice

1. What is the main purpose of using a Pipeline in machine learning?

easy

A. To combine preprocessing steps and model training into one object

B. To speed up the training by using multiple CPUs

C. To automatically select the best model type

D. To visualize the model's decision boundaries

Solution

1. Step 1: Understand what a Pipeline does

   A Pipeline chains preprocessing and model training steps so they run together smoothly.

2. Step 2: Identify the main benefit

   This chaining helps avoid mistakes and makes code cleaner by combining steps into one object.

3. Final Answer:

   To combine preprocessing steps and model training into one object -> Option A

4. Quick Check:

   Pipeline = combine steps [OK]

Hint: Pipeline bundles steps to simplify workflow [OK]

Common Mistakes:

• Thinking Pipeline speeds up training automatically
• Confusing Pipeline with model selection
• Believing Pipeline creates visualizations

2. Which syntax correctly sets the parameter n_estimators of a RandomForest inside a pipeline named pipe for GridSearchCV?

easy

A. {'randomforest-n_estimators': [10, 50, 100]}

B. {'random_forest__n_estimators': [10, 50, 100]}

C. {'randomforest.n_estimators': [10, 50, 100]}

D. {'randomforest__n_estimators': [10, 50, 100]}

Solution

1. Step 1: Recall parameter naming in Pipeline

   Parameters inside a pipeline step use double underscores: stepname__paramname.

2. Step 2: Match step name and parameter

   If the step is named 'randomforest', then 'randomforest__n_estimators' is correct syntax.

3. Final Answer:

   {'randomforest__n_estimators': [10, 50, 100]} -> Option D

4. Quick Check:

   Use double underscores between step and param [OK]

Hint: Use double underscores between step and parameter [OK]

Common Mistakes:

• Using single underscore instead of double
• Using dot or dash instead of double underscore
• Misspelling the pipeline step name

3. Given the code below, what will grid.best_params_ output?

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV

pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', RandomForestClassifier(random_state=42))
])

param_grid = {'clf__n_estimators': [20], 'clf__max_depth': [4]}
grid = GridSearchCV(pipe, param_grid, cv=2)
grid.fit(X_train, y_train)

print(grid.best_params_)

medium

A. SyntaxError due to param_grid keys

B. {'clf__n_estimators': 10, 'clf__max_depth': 2}

C. {'clf__n_estimators': 20, 'clf__max_depth': 4}

D. KeyError because 'clf' is not a pipeline step

Solution

1. Step 1: Understand pipeline and param_grid

   The pipeline has a step named 'clf' for RandomForestClassifier. The param_grid uses 'clf__' prefix correctly.

2. Step 2: Determine the output

   Since param_grid specifies only one combination, GridSearchCV will select {'clf__n_estimators': 20, 'clf__max_depth': 4} as the best parameters.

3. Final Answer:

   {'clf__n_estimators': 20, 'clf__max_depth': 4} -> Option C

4. Quick Check:

   Best params match the only tested values [OK]

Hint: With single param values, they become best_params_ [OK]

Common Mistakes:

• Confusing step name 'clf' with 'classifier'
• Using single underscore in param_grid keys
• Assuming syntax error without checking keys

4. Identify the error in this pipeline and GridSearchCV setup:

pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('model', RandomForestClassifier())
])

param_grid = {'randomforest__n_estimators': [10, 50]}
grid = GridSearchCV(pipe, param_grid)
grid.fit(X_train, y_train)

medium

A. The param_grid key should be 'model__n_estimators', not 'randomforest__n_estimators'

B. RandomForestClassifier cannot be used inside a pipeline

C. StandardScaler should not be the first step

D. GridSearchCV requires cv parameter

Solution

1. Step 1: Check pipeline step names

   The pipeline step for RandomForestClassifier is named 'model', not 'randomforest'.

2. Step 2: Match param_grid keys to pipeline steps

   Parameter keys must use the step name 'model' with double underscores, so 'model__n_estimators' is correct.

3. Final Answer:

   The param_grid key should be 'model__n_estimators', not 'randomforest__n_estimators' -> Option A

4. Quick Check:

   Param keys must match pipeline step names [OK]

Hint: Param keys must match pipeline step names exactly [OK]

Common Mistakes:

• Using wrong step name in param_grid keys
• Thinking RandomForest can't be in pipeline
• Believing cv is mandatory (it defaults to 5)

5. You want to tune both a scaler and a classifier in a pipeline using GridSearchCV. Which param_grid correctly tests StandardScaler with and without scaling, and RandomForest with 10 or 50 trees?

pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', RandomForestClassifier(random_state=0))
])

param_grid = ?

hard

A. {'scaler__': [StandardScaler(), None], 'clf__n_estimators': [10, 50]}

B. {'scaler': [StandardScaler(), None], 'clf__n_estimators': [10, 50]}

C. {'scaler': [StandardScaler(), None], 'clf__n_estimators': [10, 50], 'clf__max_depth': [None]}

D. {'scaler__with_mean': [True, False], 'clf__n_estimators': [10, 50]}

Solution

1. Step 1: Understand how to toggle scaler on/off in pipeline

   To test with and without scaling, replace the scaler step with StandardScaler() or None in param_grid using the step name 'scaler'.

2. Step 2: Set classifier parameters correctly

   Use 'clf__n_estimators' to test 10 and 50 trees for the RandomForestClassifier step named 'clf'.

3. Final Answer:

   {'scaler': [StandardScaler(), None], 'clf__n_estimators': [10, 50]} -> Option B

4. Quick Check:

   Toggle scaler by replacing step, tune clf params with double underscores [OK]

Hint: Toggle steps by replacing with None in param_grid [OK]

Common Mistakes:

• Trying to set scaler params with double underscores incorrectly
• Using 'scaler__' key with no param name
• Not using None to disable a step