Precision-recall curves in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Precision-Recall Curve Purpose

What does a precision-recall curve primarily help you understand about a classification model?

A The speed at which the model makes predictions.

B The overall accuracy of the model across all classes.

C The trade-off between the model's ability to identify positive cases and its accuracy on those predictions.

D The model's performance on negative class predictions only.

💡 Hint

Think about what precision and recall measure in classification.

Precision-recall curves show how precision and recall change with different thresholds, highlighting the balance between correctly identifying positives and avoiding false positives.

❓ Predict Output

intermediate

Output of TensorFlow Precision-Recall Calculation

What is the output shape of the precision and recall tensors when using TensorFlow's tf.metrics.PrecisionAtRecall with a batch of 5 predictions?

TensorFlow

import tensorflow as tf

labels = tf.constant([1, 0, 1, 1, 0], dtype=tf.int32)
predictions = tf.constant([0.9, 0.1, 0.8, 0.4, 0.3], dtype=tf.float32)

metric = tf.keras.metrics.PrecisionAtRecall(recall=0.8)
metric.update_state(labels, predictions)
result = metric.result()
print(result.shape)

A () # Scalar tensor representing precision value

B (5,) # Tensor with precision for each prediction

C (2,) # Tensor with precision and recall values

D (None,) # Tensor with undefined shape

💡 Hint

Check the documentation for PrecisionAtRecall output.

The PrecisionAtRecall metric returns a single scalar value representing precision at the specified recall level.

❓ Model Choice

advanced

Choosing Model Output for Precision-Recall Curve

Which type of model output is most appropriate to generate a precision-recall curve?

A Raw logits (unnormalized scores) from the last layer

B Probability scores (values between 0 and 1) for the positive class

C Binary class labels (0 or 1) predicted by the model

D One-hot encoded vectors of predicted classes

💡 Hint

Precision-recall curves require a threshold to vary. Which output allows this?

Probability scores allow varying the threshold to compute precision and recall at different points, which is essential for plotting the curve.

❓ Metrics

advanced

Interpreting Area Under Precision-Recall Curve (AUPRC)

What does a higher Area Under the Precision-Recall Curve (AUPRC) indicate about a model's performance?

A The model has faster training time.

B The model has higher overall accuracy across all classes.

C The model predicts negative classes more accurately than positive classes.

D The model has better balance between precision and recall, especially on imbalanced datasets.

💡 Hint

Think about what precision and recall measure and why AUPRC is useful.

A higher AUPRC means the model maintains high precision and recall across thresholds, which is especially important when positive cases are rare.

🔧 Debug

expert

Debugging Precision-Recall Curve Calculation in TensorFlow

Given the following code snippet, what is the cause of the error when trying to compute precision-recall curve?

import tensorflow as tf

labels = tf.constant([1, 0, 1, 0, 1])
predictions = tf.constant([0.9, 0.2, 0.8, 0.4, 0.3])

precision, recall, thresholds = tf.metrics.precision_recall_curve(labels, predictions)
print(precision, recall, thresholds)

A <code>tf.metrics.precision_recall_curve</code> does not exist in TensorFlow; the correct function is in <code>sklearn.metrics</code>.

B The labels tensor must be float32, not int32.

C The predictions tensor must be binary labels, not probabilities.

D The function requires labels and predictions to be lists, not tensors.

💡 Hint

Check TensorFlow API for precision-recall curve functions.

TensorFlow does not have a precision_recall_curve function; it is provided by scikit-learn. TensorFlow metrics focus on scalar metrics, not curve outputs.

Practice

1. What does a precision-recall curve primarily show in model evaluation?

easy

A. The loss value during training

B. The trade-off between precision and recall for different thresholds

C. The accuracy of the model on the test set

D. The number of false negatives only

Solution

1. Step 1: Understand precision and recall

   Precision measures correct positive predictions, recall measures how many actual positives are found.

2. Step 2: Understand what the curve shows

   The precision-recall curve plots precision against recall at different classification thresholds, showing their trade-off.

3. Final Answer:

   The trade-off between precision and recall for different thresholds -> Option B

4. Quick Check:

   Precision-recall curve = trade-off between precision and recall [OK]

Hint: Precision-recall curves show precision vs recall trade-off [OK]

Common Mistakes:

• Confusing precision-recall curve with accuracy plot
• Thinking it shows loss values
• Assuming it only shows false negatives

2. Which TensorFlow function is used to compute precision and recall values for plotting a precision-recall curve?

easy

A. tf.keras.metrics.Precision() and tf.keras.metrics.Recall()

B. tf.keras.losses.MeanSquaredError()

C. tf.data.Dataset.batch()

D. tf.random.uniform()

Solution

1. Step 1: Identify functions related to precision and recall

   TensorFlow provides tf.keras.metrics.Precision() and tf.keras.metrics.Recall() to compute these metrics.

2. Step 2: Exclude unrelated functions

   MeanSquaredError is a loss, Dataset.batch is for data batching, random.uniform generates random numbers.

3. Final Answer:

   tf.keras.metrics.Precision() and tf.keras.metrics.Recall() -> Option A

4. Quick Check:

   Precision and recall functions = tf.keras.metrics.Precision and tf.keras.metrics.Recall [OK]

Hint: Use tf.keras.metrics.Precision and tf.keras.metrics.Recall for these metrics [OK]

Common Mistakes:

• Using loss functions instead of metric functions
• Confusing data pipeline functions with metrics
• Using random functions for metrics

3. Given the following TensorFlow code snippet, what will be the shape of the precision and recall tensors?

import tensorflow as tf
from sklearn.metrics import precision_recall_curve
y_true = tf.constant([0, 1, 1, 0, 1])
y_scores = tf.constant([0.1, 0.4, 0.35, 0.8, 0.7])
precision, recall, thresholds = precision_recall_curve(y_true, y_scores)
print(precision.shape, recall.shape)

medium

A. (1,) (1,)

B. (5,) (5,)

C. (N,) (N,) where N is number of thresholds + 1

D. Scalar values

Solution

1. Step 1: Understand precision_recall_curve output

   The function returns precision and recall arrays for each threshold plus one extra point, so length is number of thresholds + 1.

2. Step 2: Analyze shapes

   Since thresholds vary, precision and recall are vectors, not scalars or fixed length equal to input size.

3. Final Answer:

   (N,) (N,) where N is number of thresholds + 1 -> Option C

4. Quick Check:

   Precision and recall shapes = vectors with length thresholds+1 [OK]

Hint: precision_recall_curve returns arrays, not scalars [OK]

Common Mistakes:

• Assuming output shapes match input size
• Expecting scalar precision and recall
• Confusing thresholds count with input length

4. The following code attempts to plot a precision-recall curve but raises an error. What is the likely cause?

import tensorflow as tf
import matplotlib.pyplot as plt

y_true = tf.constant([0, 1, 1, 0])
y_scores = tf.constant([0.2, 0.8, 0.6, 0.4])
precision, recall, thresholds = tf.metrics.PrecisionRecallCurve()(y_true, y_scores)
plt.plot(precision, recall)
plt.show()

medium

A. Precision and recall are swapped in the plot call

B. plt.plot requires thresholds as x-axis

C. y_true and y_scores must be numpy arrays, not tensors

D. tf.metrics.PrecisionRecallCurve() does not exist

Solution

1. Step 1: Check TensorFlow API correctness

   TensorFlow does not have tf.metrics.PrecisionRecallCurve(); typically use from sklearn.metrics import precision_recall_curve.

2. Step 2: Understand error cause

   Using a non-existent function causes an AttributeError before plotting.

3. Final Answer:

   tf.metrics.PrecisionRecallCurve() does not exist -> Option D

4. Quick Check:

   Correct approach is sklearn.metrics.precision_recall_curve [OK]

Hint: Use sklearn.metrics.precision_recall_curve, not tf.metrics [OK]

Common Mistakes:

• Using non-existent TensorFlow function for PR curve
• Swapping precision and recall axes
• Assuming tensors can't be plotted

5. You have a highly imbalanced dataset with 1% positive class. You want to evaluate your TensorFlow model's ability to detect positives. Which approach is best to use and why? A) Use accuracy metric only B) Use confusion matrix without thresholds C) Use loss value during training D) Use precision-recall curve to evaluate model

hard

A. Use precision-recall curve to evaluate model

B. Use confusion matrix without thresholds

C. Use loss value during training

D. Use accuracy metric only

Solution

1. Step 1: Understand dataset imbalance impact

   With 1% positives, accuracy can be misleading because predicting all negatives gives high accuracy.

2. Step 2: Choose appropriate evaluation metric

   Precision-recall curves focus on positive class detection and show trade-offs, ideal for imbalanced data.

3. Final Answer:

   Use precision-recall curve to evaluate model -> Option A

4. Quick Check:

   Imbalanced data = use precision-recall curve [OK]

Hint: For imbalanced data, prefer precision-recall curves [OK]

Common Mistakes:

• Relying on accuracy for imbalanced data
• Ignoring threshold effects in confusion matrix
• Using loss value as evaluation metric