Model evaluation and metrics in Apache Spark - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to calculate the accuracy of a classification model using Spark MLlib.

Apache Spark

from pyspark.ml.evaluation import MulticlassClassificationEvaluator

evaluator = MulticlassClassificationEvaluator(labelCol="label", predictionCol="prediction", metricName=[1])
accuracy = evaluator.evaluate(predictions)
print(f"Accuracy: {accuracy}")

Drag options to blanks, or click blank then click option'

A"f1"

B"weightedPrecision"

C"accuracy"

D"areaUnderROC"

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'areaUnderROC' which is for binary classifiers only.

Confusing 'f1' score with accuracy.

The metricName parameter should be set to "accuracy" to calculate the accuracy score of the classification model.

2fill in blank

medium

Complete the code to compute the root mean squared error (RMSE) for a regression model in Spark.

Apache Spark

from pyspark.ml.evaluation import RegressionEvaluator

evaluator = RegressionEvaluator(labelCol="label", predictionCol="prediction", metricName=[1])
rmse = evaluator.evaluate(predictions)
print(f"RMSE: {rmse}")

Drag options to blanks, or click blank then click option'

A"r2"

B"rmse"

C"mse"

D"mae"

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'mse' which is mean squared error, not root mean squared error.

Using 'r2' which measures variance explained, not error.

To calculate root mean squared error, set metricName to "rmse" in RegressionEvaluator.

3fill in blank

hard

Fix the error in the code to compute the area under the ROC curve for a binary classification model.

Apache Spark

from pyspark.ml.evaluation import BinaryClassificationEvaluator

evaluator = BinaryClassificationEvaluator(labelCol="label", rawPredictionCol=[1], metricName="areaUnderROC")
auc = evaluator.evaluate(predictions)
print(f"AUC: {auc}")

Drag options to blanks, or click blank then click option'

A"rawPrediction"

B"prediction"

C"probability"

D"score"

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'prediction' column which contains discrete labels, not scores.

Using 'probability' column which is a vector, not a single score.

The rawPredictionCol should be set to "rawPrediction" because the evaluator expects raw prediction scores for ROC calculation.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that maps each word to its length only if the length is greater than 3.

Apache Spark

words = ["apple", "cat", "banana", "dog"]
lengths = { [1] : [2] for word in words if len(word) > 3 }

Drag options to blanks, or click blank then click option'

Aword

Blen(word)

Cword.upper()

Dlen

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'len' as a value instead of calling len(word).

Using 'word.upper()' which changes the word but is not length.

The dictionary comprehension maps each word (key) to its length (value) for words longer than 3 characters.

5fill in blank

hard

Fill all three blanks to create a dictionary comprehension that maps each uppercase word to its length only if the length is greater than 3.

Apache Spark

words = ["apple", "cat", "banana", "dog"]
lengths = { [1] : [2] for word in words if [3] 3 }

Drag options to blanks, or click blank then click option'

Aword.upper()

Blen(word)

C>

Dword

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'word' instead of 'word.upper()' for the key.

Using '<' instead of '>' in the condition.

The dictionary comprehension maps each uppercase word to its length only if the length is greater than 3.

Practice

1. Which metric is commonly used to evaluate a classification model's accuracy in Apache Spark?

easy

A. R-squared

B. Root Mean Squared Error

C. Accuracy

D. Mean Absolute Error

Solution

1. Step 1: Understand classification metrics

   Classification models predict categories, so accuracy measures correct predictions over total predictions.

2. Step 2: Match metric to model type

   Accuracy is the standard metric for classification, while RMSE, MAE, and R-squared are for regression.

3. Final Answer:

   Accuracy -> Option C

4. Quick Check:

   Classification accuracy = Accuracy [OK]

Hint: Use accuracy for classification, RMSE for regression [OK]

Common Mistakes:

• Confusing regression metrics with classification
• Using R-squared for classification
• Choosing MAE for classification

2. Which of the following is the correct way to create a BinaryClassificationEvaluator in Spark to evaluate areaUnderROC?

easy

A. BinaryClassificationEvaluator(metric="areaUnderROC")

B. BinaryClassificationEvaluator(metricName="areaUnderROC")

C. BinaryClassificationEvaluator(metricName='accuracy')

D. BinaryClassificationEvaluator(metric='accuracy')

Solution

1. Step 1: Recall evaluator syntax

   BinaryClassificationEvaluator uses parameter metricName to specify the metric.

2. Step 2: Check metric value

   "areaUnderROC" is a valid metricName; accuracy is not valid for this evaluator.

3. Final Answer:

   BinaryClassificationEvaluator(metricName="areaUnderROC") -> Option B

4. Quick Check:

   Use metricName="areaUnderROC" in BinaryClassificationEvaluator [OK]

Hint: Use metricName, not metric, for evaluator metrics [OK]

Common Mistakes:

• Using 'metric' instead of 'metricName'
• Passing invalid metric names like 'accuracy'
• Using single quotes inconsistently

3. Given the following Spark code, what will be the output of the evaluator?

from pyspark.ml.evaluation import RegressionEvaluator

predictions = spark.createDataFrame([
  (3.0, 2.5),
  (5.0, 5.0),
  (2.5, 3.0),
  (7.0, 8.0)
], ["label", "prediction"])

evaluator = RegressionEvaluator(metricName="rmse")
rmse = evaluator.evaluate(predictions)
print(round(rmse, 2))

medium

A. 1.00

B. 0.50

C. 0.75

D. 0.61

Solution

1. Step 1: Calculate squared errors

   Errors: (3.0-2.5)^2=0.25, (5.0-5.0)^2=0, (2.5-3.0)^2=0.25, (7.0-8.0)^2=1.0

2. Step 2: Compute RMSE

   Mean squared error = (0.25+0+0.25+1.0)/4 = 0.375; RMSE = sqrt(0.375) ≈ 0.612

3. Final Answer:

   0.61 -> Option D

4. Quick Check:

   RMSE ≈ 0.61 [OK]

Hint: RMSE is sqrt of average squared error [OK]

Common Mistakes:

• Calculating mean absolute error instead of RMSE
• Forgetting to take square root
• Rounding errors

4. You run this Spark code but get an error:

from pyspark.ml.evaluation import RegressionEvaluator

evaluator = RegressionEvaluator(metricName="accuracy")

What is the cause of the error?

medium

A. "accuracy" is not a valid metric for RegressionEvaluator

B. RegressionEvaluator requires a DataFrame input

C. metricName parameter is misspelled

D. RegressionEvaluator cannot be imported from pyspark.ml.evaluation

Solution

1. Step 1: Check valid metrics for RegressionEvaluator

   RegressionEvaluator supports metrics like rmse, mse, mae, r2 but not accuracy.

2. Step 2: Identify error cause

   Using "accuracy" causes an error because it is for classification, not regression.

3. Final Answer:

   "accuracy" is not a valid metric for RegressionEvaluator -> Option A

4. Quick Check:

   RegressionEvaluator metrics exclude accuracy [OK]

Hint: Use classification evaluator for accuracy metric [OK]

Common Mistakes:

• Confusing metric names between evaluators
• Assuming metricName typo causes error
• Ignoring evaluator import errors

5. You have a Spark ML model for binary classification. You want to evaluate it using both areaUnderROC and F1 score. Which approach correctly evaluates both metrics on the same predictions DataFrame?

hard

A. Use BinaryClassificationEvaluator with metricName="areaUnderROC" and MulticlassClassificationEvaluator with metricName="f1"

B. Use BinaryClassificationEvaluator twice with metricName="areaUnderROC" and metricName="f1" respectively

C. Use RegressionEvaluator with metricName="areaUnderROC" and MulticlassClassificationEvaluator with metricName="f1"

D. Use MulticlassClassificationEvaluator twice with metricName="areaUnderROC" and metricName="f1"

Solution

1. Step 1: Identify evaluators for each metric

   areaUnderROC is supported by BinaryClassificationEvaluator; F1 score is supported by MulticlassClassificationEvaluator.

2. Step 2: Combine evaluators correctly

   Use BinaryClassificationEvaluator for areaUnderROC and MulticlassClassificationEvaluator for F1 on the same predictions DataFrame.

3. Final Answer:

   Use BinaryClassificationEvaluator with metricName="areaUnderROC" and MulticlassClassificationEvaluator with metricName="f1" -> Option A

4. Quick Check:

   BinaryClassificationEvaluator + MulticlassClassificationEvaluator = correct combo [OK]

Hint: Use correct evaluator per metric type [OK]

Common Mistakes:

• Using RegressionEvaluator for classification metrics
• Using BinaryClassificationEvaluator for F1 score
• Using MulticlassClassificationEvaluator for areaUnderROC