Hyperparameter tuning with CrossValidator in Apache Spark - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of CrossValidator with ParamGridBuilder

What will be the output type of the following Spark ML pipeline after fitting with CrossValidator?

Apache Spark

from pyspark.ml.classification import LogisticRegression
from pyspark.ml.tuning import CrossValidator, ParamGridBuilder
from pyspark.ml.evaluation import BinaryClassificationEvaluator

lr = LogisticRegression()
paramGrid = ParamGridBuilder()\
    .addGrid(lr.regParam, [0.1, 0.01])\
    .addGrid(lr.elasticNetParam, [0.0, 0.5, 1.0])\
    .build()

cv = CrossValidator(estimator=lr,
                    estimatorParamMaps=paramGrid,
                    evaluator=BinaryClassificationEvaluator(),
                    numFolds=3)

cvModel = cv.fit(trainingData)

print(type(cvModel))

A <class 'pyspark.ml.PipelineModel'>

B <class 'pyspark.ml.tuning.CrossValidatorModel'>

C <class 'pyspark.ml.classification.LogisticRegressionModel'>

D <class 'pyspark.ml.tuning.ParamGridBuilder'>

💡 Hint

CrossValidator.fit returns a model that contains the best model and metrics.

The fit method of CrossValidator returns a CrossValidatorModel object, which holds the best model found during tuning and the metrics for all parameter combinations.

❓ data_output

intermediate

Number of models trained in CrossValidator

Given a ParamGrid with 3 values for regParam and 2 values for maxIter, and numFolds=4 in CrossValidator, how many models will be trained during fitting?

Apache Spark

from pyspark.ml.classification import LogisticRegression
from pyspark.ml.tuning import CrossValidator, ParamGridBuilder

lr = LogisticRegression()
paramGrid = ParamGridBuilder()\
    .addGrid(lr.regParam, [0.01, 0.1, 1.0])\
    .addGrid(lr.maxIter, [10, 20])\
    .build()

cv = CrossValidator(estimator=lr,
                    estimatorParamMaps=paramGrid,
                    evaluator=None,
                    numFolds=4)

A 8

B 12

C 24

D 6

💡 Hint

Multiply number of parameter combinations by number of folds.

There are 3 regParam values and 2 maxIter values, so 3*2=6 parameter combinations. With 4 folds, total models trained = 6*4 = 24.

🔧 Debug

advanced

Identify the error in CrossValidator setup

What error will the following code raise when running cv.fit(trainingData)?

Apache Spark

from pyspark.ml.classification import LogisticRegression
from pyspark.ml.tuning import CrossValidator, ParamGridBuilder
from pyspark.ml.evaluation import BinaryClassificationEvaluator

lr = LogisticRegression()
paramGrid = ParamGridBuilder()\
    .addGrid(lr.regParam, [0.1, 0.01])\
    .build()

cv = CrossValidator(estimator=lr,
                    estimatorParamMaps=paramGrid,
                    evaluator=BinaryClassificationEvaluator(),
                    numFolds=0)

cvModel = cv.fit(trainingData)

A ValueError: numFolds must be at least 2

B TypeError: evaluator must be a string

C AttributeError: 'LogisticRegression' object has no attribute 'fit'

D No error, runs successfully

💡 Hint

CrossValidator requires at least 2 folds for cross-validation.

numFolds=0 is invalid because cross-validation requires at least 2 folds. Spark raises a ValueError indicating this.

🚀 Application

advanced

Choosing the best model from CrossValidatorModel

After fitting a CrossValidator, how can you extract the best model to use for predictions?

Apache Spark

cvModel = cv.fit(trainingData)
bestModel = ???

A cvModel.model

B cvModel.getBestModel()

C cvModel.best_model

D cvModel.bestModel

💡 Hint

Check the attribute that stores the best fitted model in CrossValidatorModel.

The best fitted model is stored in the attribute bestModel of the CrossValidatorModel object.

🧠 Conceptual

expert

Effect of evaluator metric on CrossValidator results

Which statement best describes how the evaluator metric affects the CrossValidator's model selection?

A The evaluator metric determines how models are scored and which model is chosen as best.

B The evaluator metric is ignored if numFolds is greater than 3.

C The evaluator metric only affects the training speed, not model selection.

D The evaluator metric is used to generate new hyperparameters automatically.

💡 Hint

Think about what role the evaluator plays in cross-validation.

The evaluator computes a metric (like accuracy or AUC) on validation folds. CrossValidator uses this metric to pick the best model.

Practice

1. What is the main purpose of using CrossValidator in Apache Spark's machine learning pipeline?

easy

A. To visualize the model's performance metrics

B. To train the model on the entire dataset without validation

C. To clean and preprocess the input data

D. To find the best hyperparameters by testing multiple data splits

Solution

1. Step 1: Understand CrossValidator's role

   CrossValidator splits data into training and validation sets multiple times to evaluate model performance.

2. Step 2: Identify the purpose of hyperparameter tuning

   It tests different hyperparameter combinations to find the best model settings.

3. Final Answer:

   To find the best hyperparameters by testing multiple data splits -> Option D

4. Quick Check:

   CrossValidator = Best hyperparameters [OK]

Hint: CrossValidator tests parameters on data splits to find best model [OK]

Common Mistakes:

• Thinking CrossValidator trains on full data without validation
• Confusing CrossValidator with data cleaning tools
• Assuming CrossValidator is for visualization only

2. Which of the following is the correct way to create a parameter grid using ParamGridBuilder in Spark ML?

easy

A. paramGrid = ParamGridBuilder().addGrid(model.maxDepth, [2, 5]).build()

B. paramGrid = ParamGridBuilder.addGrid(model.maxDepth, [2, 5]).build()

C. paramGrid = ParamGridBuilder().addGrid(model.maxDepth = [2, 5]).build()

D. paramGrid = ParamGridBuilder().addGrid([2, 5], model.maxDepth).build()

Solution

1. Step 1: Recall ParamGridBuilder syntax

   ParamGridBuilder is instantiated, then addGrid is called with parameter and list of values.

2. Step 2: Check each option's syntax

   paramGrid = ParamGridBuilder().addGrid(model.maxDepth, [2, 5]).build() correctly calls ParamGridBuilder() then addGrid(model.maxDepth, [2, 5]) and finally build().

3. Final Answer:

   paramGrid = ParamGridBuilder().addGrid(model.maxDepth, [2, 5]).build() -> Option A

4. Quick Check:

   ParamGridBuilder() + addGrid(param, values) + build() [OK]

Hint: Use ParamGridBuilder() then addGrid(param, values) then build() [OK]

Common Mistakes:

• Calling addGrid without parentheses on ParamGridBuilder
• Using assignment inside addGrid instead of parameters
• Swapping parameter and values order in addGrid

3. Given the following code snippet, what will be the output of cvModel.bestModel.getMaxDepth()?

from pyspark.ml.classification import DecisionTreeClassifier
from pyspark.ml.tuning import CrossValidator, ParamGridBuilder
from pyspark.ml.evaluation import BinaryClassificationEvaluator

model = DecisionTreeClassifier()
paramGrid = ParamGridBuilder().addGrid(model.maxDepth, [2, 4]).build()
evaluator = BinaryClassificationEvaluator()

cv = CrossValidator(estimator=model, estimatorParamMaps=paramGrid, evaluator=evaluator, numFolds=2)
cvModel = cv.fit(trainingData)

best_depth = cvModel.bestModel.getMaxDepth()

medium

A. The average of 2 and 4, which is 3

B. 2 or 4 depending on which depth gave better validation metric

C. An error because getMaxDepth() is not a valid method

D. Always 2 because it is the first value in the grid

Solution

1. Step 1: Understand CrossValidator fitting

   CrossValidator tries all parameter combinations and selects the best model based on evaluator metric.

2. Step 2: Interpret bestModel.getMaxDepth()

   The bestModel will have maxDepth set to the value that gave the best validation score, either 2 or 4.

3. Final Answer:

   2 or 4 depending on which depth gave better validation metric -> Option B

4. Quick Check:

   bestModel maxDepth = best param tested [OK]

Hint: bestModel has hyperparameter with best validation score [OK]

Common Mistakes:

• Assuming bestModel always uses first grid value
• Thinking getMaxDepth() method does not exist
• Calculating average of grid values instead of selecting best

4. Identify the error in this code snippet for hyperparameter tuning with CrossValidator:

from pyspark.ml.classification import LogisticRegression
from pyspark.ml.tuning import CrossValidator, ParamGridBuilder
from pyspark.ml.evaluation import BinaryClassificationEvaluator

lr = LogisticRegression()
paramGrid = ParamGridBuilder().addGrid(lr.regParam, [0.1, 0.01])

evaluator = BinaryClassificationEvaluator()
cv = CrossValidator(estimator=lr, estimatorParamMaps=paramGrid, evaluator=evaluator, numFolds=3)
cvModel = cv.fit(trainingData)

medium

A. Missing .build() call after ParamGridBuilder()

B. BinaryClassificationEvaluator cannot be used with LogisticRegression

C. numFolds must be at least 5 for CrossValidator

D. regParam values must be strings, not floats

Solution

1. Step 1: Check ParamGridBuilder usage

   ParamGridBuilder requires calling .build() to create the parameter grid list.

2. Step 2: Identify missing method call

   The code misses .build() after addGrid, so paramGrid is not a valid parameter map list.

3. Final Answer:

   Missing .build() call after ParamGridBuilder() -> Option A

4. Quick Check:

   ParamGridBuilder().addGrid(...).build() required [OK]

Hint: Always end ParamGridBuilder with .build() to create grid [OK]

Common Mistakes:

• Forgetting to call .build() after addGrid
• Using wrong evaluator for classification
• Setting numFolds less than 2 (valid but uncommon)

5. You want to tune two hyperparameters of a RandomForestClassifier: numTrees with values [10, 20] and maxDepth with values [5, 10]. How many total models will CrossValidator train if numFolds=3?

hard

A. 30 models

B. 6 models

C. 12 models

D. 60 models

Solution

1. Step 1: Calculate total hyperparameter combinations

   numTrees has 2 values, maxDepth has 2 values, so total combinations = 2 * 2 = 4.

2. Step 2: Multiply by number of folds

   CrossValidator trains each combination on each fold, so total models = 4 * 3 = 12.

3. Step 3: Re-check calculation

   Wait, the question asks how many total models trained. Each fold trains one model per combination, so total = combinations * folds = 4 * 3 = 12.

4. Final Answer:

   12 models -> Option C

5. Quick Check:

   Combinations * folds = 12 [OK]

Hint: Multiply parameter combos by folds for total models trained [OK]

Common Mistakes:

• Multiplying number of values incorrectly
• Forgetting to multiply by number of folds
• Confusing total models with total parameter combos