Bird
Raised Fist0

Implement ROC Curve and AUC from Predictions

CODINGintermediateInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
▶
Execution Trace
Step 1/5
Threshold = 0.5

TPR = 0.8, FPR = 0.2 at threshold 0.5

🎯 Understanding how threshold affects TPR and FPR is crucial for ROC analysis.
from sklearn.metrics import roc_curve
fpr, tpr, thresholds = roc_curve(y_true, y_scores)
threshold = 0.5
tpr_at_threshold = tpr[thresholds >= threshold][0]
fpr_at_threshold = fpr[thresholds >= threshold][0]
Threshold = 0.7

TPR decreases to 0.6, FPR decreases to 0.1 at threshold 0.7

🎯 Higher thresholds generally decrease both TPR and FPR.
threshold = 0.7
tpr_at_threshold = tpr[thresholds >= threshold][0]
fpr_at_threshold = fpr[thresholds >= threshold][0]
AUC Calculation

AUC computed as 0.85, indicating good model performance.

🎯 AUC is a single scalar value summarizing the ROC curve.
from sklearn.metrics import auc
roc_auc = auc(fpr, tpr)
Threshold = 0.3

TPR increases to 0.9, FPR increases to 0.4 at threshold 0.3

🎯 Lower thresholds increase both TPR and FPR, showing more aggressive classification.
threshold = 0.3
tpr_at_threshold = tpr[thresholds >= threshold][0]
fpr_at_threshold = fpr[thresholds >= threshold][0]
Optimal Threshold

Optimal threshold balances TPR and FPR for best performance.

🎯 Choosing the right threshold is key for model deployment.
optimal_idx = np.argmax(tpr - fpr)
optimal_threshold = thresholds[optimal_idx]
📊
Diagram
Diagram
┌──────────────────────────────────────────────────────────┐
│                        ROC Curve Plot                    │
├──────────────────────────────────────────────────────────┤
│                                                          │
│  1.0 ────────────────────────────────────────────────────│
│      │*                                                 *│
│  0.8 │ *                                               * │
│      │  *                                             *  │
│  0.6 │   *                                           *   │
│      │    *                                         *    │
│  0.4 │     *                                       *     │
│      │      *                                     *      │
│  0.2 │       *                                   *       │
│      │        *                                 *        │
│  0.0 └─────────────────────────────────────────*─────────│
│      0.0       0.2       0.4       0.6       0.8       1.0│
│                                                          │
└──────────────────────────────────────────────────────────┘
Labels
  • ROC Curve: Graph of TPR vs FPR
  • TPR: True Positive Rate (Sensitivity)
  • FPR: False Positive Rate (1 - Specificity)
  • AUC: Area Under the ROC Curve
  • Thresholds: Points on the curve
🔑
Key Moments
▲
Q: Why does the ROC curve start at (0,0) and end at (1,1)?
A: The ROC curve starts at (0,0) because at the lowest threshold, no positives are detected (TPR=0) and no negatives are falsely detected (FPR=0). It ends at (1,1) because at the highest threshold, all positives and negatives are detected (TPR=1, FPR=1).
Q: How does the AUC value relate to model performance?
A: AUC represents the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance. AUC values closer to 1 indicate better model performance.
Q: What does a point on the ROC curve represent?
A: Each point on the ROC curve represents a TPR and FPR pair at a specific threshold, showing the trade-off between sensitivity and specificity.
Trace Summary

This diagram and trace illustrate the implementation of the ROC curve and AUC calculation from predictions. The ROC curve graphically represents the trade-off between true positive rate (TPR) and false positive rate (FPR) at various thresholds. The AUC quantifies the overall ability of the model to discriminate between positive and negative classes. Key steps include calculating TPR and FPR at different thresholds, computing the AUC, and identifying the optimal threshold for model performance. Understanding these concepts is crucial for evaluating classification models effectively.

Primary Focus

✓ Implementing and understanding ROC Curve and AUC calculation

Practice

(1/5)
1. What does the following Python code snippet output when implementing the K-Means clustering algorithm from scratch? 
import numpy as np

# Sample data
X = np.array([[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]])

# Initial centroids
centroids = np.array([[1, 2], [10, 2]])

# Calculate distances
for i in range(len(X)):
    distances = np.linalg.norm(X[i] - centroids, axis=1)
    print(distances)
easy
A. Prints the distances between each point and the second centroid only.
B. Prints the pairwise distances between each point and both centroids.
C. Prints the distances between each point and the first centroid only.
D. Prints the sum of distances for each point to both centroids.

Solution

  1. Step 1: Understand the code logic

    The code calculates the Euclidean distance from each data point to both centroids.

  2. Step 2: Eliminate wrong options

    Prints the distances between each point and the first centroid only. and C are incorrect because the code calculates distances to both centroids, not just one. Prints the sum of distances for each point to both centroids. is incorrect because it doesn't sum the distances.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Distances to all centroids, not just one. [OK]
Hint: Distances to all centroids, not just one. [OK]
Common Mistakes:
2. What is the time complexity of calculating precision, recall, and F1-score from a confusion matrix?
easy
A. O(1)
B. O(n)
C. O(n^2)
D. O(log n)

Solution

  1. Step 1: Analyze the operations

    Calculating precision, recall, and F1-score involves basic arithmetic operations on a fixed-size matrix.

  2. Step 2: Determine complexity

    These operations are constant time, O(1), as they do not depend on the size of the input data.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Fixed-size matrix operations are O(1). [OK]
Hint: Fixed-size matrix operations are O(1). [OK]
Common Mistakes:
3. What does the following Python code output?
import numpy as np

def softmax(x):
    e_x = np.exp(x - np.max(x))
    return e_x / e_x.sum(axis=0)

x = np.array([2.0, 1.0, 0.1])
print(softmax(x))
easy
A. [0.659, 0.242, 0.099]
B. [0.5, 0.3, 0.2]
C. [0.665, 0.244, 0.091]
D. [0.7, 0.2, 0.1]

Solution

  1. Step 1: Calculate softmax

    Compute the exponentials of the input array after subtracting the max value for numerical stability.

  2. Step 2: Normalize

    Divide each exponential by the sum of all exponentials to get probabilities.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Softmax outputs sum to 1 [OK]
Hint: Softmax outputs sum to 1 [OK]
Common Mistakes:
4. How can you optimize the backward pass of batch normalization to improve computational efficiency?
hard
A. Fuse batch normalization with preceding linear layers.
B. Pre-compute gradients for all layers.
C. Use a smaller batch size to reduce computation.
D. Use a higher learning rate to speed up convergence.

Solution

  1. Step 1: Identify optimization techniques

    Fusing operations can reduce computational overhead.

  2. Step 2: Evaluate each option

    Fusing batch normalization with linear layers reduces redundant computations.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Fuse layers to optimize batch norm [OK]
Hint: Fuse layers to optimize batch norm [OK]
Common Mistakes:
5. How can you optimize the dropout layer implementation to reduce computational overhead during training?
hard
A. Use in-place operations to apply the mask.
B. Use a fixed mask for all training iterations.
C. Precompute masks for all layers before training.
D. Reduce the dropout rate to minimize mask application.

Solution

  1. Step 1: Identify Optimization Opportunities

    In-place operations reduce memory usage and computational overhead by modifying data directly.

  2. Step 2: Eliminate Wrong Options

    Use a fixed mask for all training iterations. reduces randomness, harming model performance. Precompute masks for all layers before training. increases memory usage. Reduce the dropout rate to minimize mask application. changes model behavior, not efficiency.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    In-place operations save memory and time. [OK]
Hint: In-place operations save memory and time. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard