Bird
Raised Fist0

The Decision Tree Dilemma

CODINGintermediateInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🐛
The Decision Tree Dilemma
The issue started immediately after the deployment last Friday.

A decision tree model used for customer churn prediction in a telecom company, processing 500k records daily.

Symptom
Model accuracy dropped from 85% to 60% after the latest deployment.
📡
Signals
metric
Training accuracy remains high at 90%, but test accuracy dropped significantly.
Training accuracy: 90%, Test accuracy: 60%
log
Deployment logs show a change in the data preprocessing step.
Preprocessing step modified to handle missing values differently.
distribution
Distribution of feature 'customer_age' in training vs. test set.
Training set mean: 35, Test set mean: 45
feature_importance
Feature importance for 'customer_age' dropped significantly.
'customer_age' importance dropped from 0.25 to 0.05
code
Code review shows recent changes in the recursive split logic.
Recursive split logic modified to use median instead of mean for splits.
🔬
Hypotheses
The change in data preprocessing caused a mismatch between training and test data distributions.
Supporting Signals
logdistribution
Verify: Check if the preprocessing change affects the feature distributions differently in training and test sets.
import pandas as pd
train_data = pd.read_csv('train.csv')
test_data = pd.read_csv('test.csv')
print(train_data['customer_age'].mean(), test_data['customer_age'].mean())
The modification in recursive split logic is causing suboptimal splits.
Supporting Signals
codefeature_importance
Verify: Review the decision tree splits to see if they align with expected feature distributions.
def check_splits(data):
    # Simulate split logic
    median_split = data['customer_age'].median()
    return median_split
train_data = pd.read_csv('train.csv')
print(check_splits(train_data))
🎯
Root Cause
The recursive split logic was incorrectly modified to use median instead of mean, leading to poor splits.
Why It Happened

A recent code change intended to improve robustness inadvertently altered the split criteria.

🔧
Fix Strategy
ImmediateRollback to the previous version of the decision tree model.
Short-termRevert the recursive split logic to use mean instead of median.
Long-termImplement a robust testing framework for model logic changes, including split criteria.
🐍
Python Fix
Python
def recursive_split(data, feature):
    # Corrected split logic
    mean_split = data[feature].mean()
    left_split = data[data[feature] <= mean_split]
    right_split = data[data[feature] > mean_split]
    return left_split, right_split
✗
Common Wrong Diagnoses
▼ Show
The model is overfitting due to too many features.
Why candidates think this: High training accuracy and low test accuracy often suggest overfitting.
Why wrong: The feature importance and distribution signals indicate a problem with the split logic, not feature count.
Data leakage between training and test sets.
Why candidates think this: A sudden drop in test accuracy can suggest data leakage.
Why wrong: The distribution and preprocessing logs show a mismatch caused by preprocessing changes, not leakage.
📊
Monitoring to Add
  • Monitor feature distribution changes between training and test datasets.
  • Track feature importance changes after each deployment.
  • Set up alerts for significant drops in test accuracy post-deployment.
📋
Process Change

Require code reviews for all changes affecting model logic, with specific attention to split criteria.

🏗️
Architectural Change

Adopt a modular design for preprocessing and model logic to isolate and test changes independently.

💡
Key Insight
Changes in model logic, especially in recursive split criteria, can drastically affect decision tree performance.

Practice

(1/5)
1. 
import torch.optim as optim

model = ...  # some PyTorch model
optimizer = optim.SGD(model.parameters(), lr=0.1)
scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=30, gamma=0.1)

for epoch in range(100):
    # training loop
    scheduler.step()

print(optimizer.param_groups[0]['lr'])
What will be the learning rate after 100 epochs?
easy
A. 0.1
B. 0.001
C. 0.0001
D. 0.01

Solution

  1. Step 1: Understand StepLR behavior

    The learning rate is multiplied by gamma (0.1) every step_size (30 epochs).

  2. Step 2: Calculate the learning rate after 100 epochs

    After 100 epochs, the learning rate is updated 3 times (at epochs 30, 60, and 90): 0.1 * 0.1 * 0.1 * 0.1 = 0.001.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    StepLR reduces LR by gamma every step_size epochs. [OK]
Hint: StepLR reduces LR by gamma every step_size epochs. [OK]
Common Mistakes:
2. What is the time complexity of computing the scaled dot-product attention for a single query with n keys?
easy
A. O(n^2)
B. O(n log n)
C. O(n)
D. O(1)

Solution

  1. Step 1: Identify operations

    For a single query, the main operations are dot product and softmax, both O(n).

  2. Step 2: Eliminate incorrect complexities

    O(n^2) and O(n log n) are incorrect as they imply more complex operations than present.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Single query, n keys -> O(n) [OK]
Hint: Single query, n keys -> O(n) [OK]
Common Mistakes:
3. What does the following Python code output?
import numpy as np

def softmax(x):
    e_x = np.exp(x - np.max(x))
    return e_x / e_x.sum(axis=0)

x = np.array([2.0, 1.0, 0.1])
print(softmax(x))
easy
A. [0.659, 0.242, 0.099]
B. [0.5, 0.3, 0.2]
C. [0.665, 0.244, 0.091]
D. [0.7, 0.2, 0.1]

Solution

  1. Step 1: Calculate softmax

    Compute the exponentials of the input array after subtracting the max value for numerical stability.

  2. Step 2: Normalize

    Divide each exponential by the sum of all exponentials to get probabilities.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Softmax outputs sum to 1 [OK]
Hint: Softmax outputs sum to 1 [OK]
Common Mistakes:
4. In a 2D convolution implementation, how would you handle an input image with a single channel differently from a multi-channel image?
medium
A. Apply the same kernel to each channel separately.
B. Use a different kernel size for each channel.
C. Sum the channels before applying the kernel.
D. Concatenate the channels before applying the kernel.

Solution

  1. Step 1: Understand multi-channel convolution

    For multi-channel images, the same kernel is applied to each channel separately, and the results are summed to produce the final output.

  2. Step 2: Eliminate wrong options

    Use a different kernel size for each channel. is incorrect as kernel size remains constant. Sum the channels before applying the kernel. loses channel information. Concatenate the channels before applying the kernel. is not a standard approach.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Apply kernel to each channel, then sum. [OK]
Hint: Apply kernel to each channel, then sum. [OK]
Common Mistakes:
5. How can you optimize the following non-max suppression code for large datasets? 
def non_max_suppression(boxes, scores, iou_threshold):
    indices = np.argsort(scores)[::-1]
    keep = []
    while len(indices) > 0:
        current = indices[0]
        keep.append(current)
        indices = indices[1:]
        indices = [i for i in indices if iou(boxes[current], boxes[i]) < iou_threshold]
    return keep
hard
A. Parallelize the IoU calculations.
B. Use a more efficient sorting algorithm.
C. Reduce the IoU threshold.
D. Use a different data structure for indices.

Solution

  1. Step 1: Identify bottleneck

    IoU calculations are the most computationally expensive part.

  2. Step 2: Optimize bottleneck

    Parallelizing IoU calculations can significantly reduce runtime for large datasets.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Parallelize expensive operations [OK]
Hint: Parallelize expensive operations [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard