Bird
Raised Fist0

Why Machine Learning Exists and What Rules Cannot Solve

THEORYbeginnerInterview freq: medium
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🎯
Designing a System to Predict Customer Churn
40 min · junior
junior

You are part of a team tasked with designing a system to predict customer churn for a subscription-based service with 1M active users. The company has historical data on user activity, subscription renewals, and customer support interactions. The system must provide predictions with a latency of under 200ms for real-time decision-making. The team consists of 3 engineers, and the project timeline is 4 months.

Given Info
  • 1M active users
  • Historical data on user activity, renewals, and support interactions
  • Latency requirement of under 200ms
  • Team of 3 engineers
  • 4-month project timeline
📋
Guided Steps
1/5
Step 1
Understanding the Problem
What are the limitations of using rule-based systems for predicting customer churn in this scenario?
💡 Hint
Consider the complexity and variability of user behavior and how rules might struggle to capture this.
Key Decision
Machine learning because user behavior is too complex and variable for static rules.
Interviewer Expectation
Mid: Identifies that rules can't capture complex patterns. Senior: Explains how ML models can adapt to new patterns over time.
⚠️Common Wrong Answer
Proposing a rule-based system, which would fail to adapt to changing user behavior and miss subtle patterns.
Step 2
Choosing the Right Data
Which data sources should be prioritized for building the churn prediction model?
💡 Hint
Think about which data sources are most indicative of churn behavior.
Key Decision
Prioritize user activity and support interactions because they directly reflect engagement and dissatisfaction.
Interviewer Expectation
Mid: Selects relevant data sources. Senior: Justifies choices with examples of how each data source can indicate churn.
⚠️Common Wrong Answer
Ignoring support interactions, which are critical for understanding dissatisfaction.
Step 3
Model Selection
What type of machine learning model would be suitable given the constraints?
💡 Hint
Consider the trade-off between model complexity and inference speed.
Key Decision
Use a lightweight model like logistic regression or decision trees for fast inference.
Interviewer Expectation
Mid: Chooses a simple model for speed. Senior: Discusses trade-offs and potential for more complex models if latency allows.
⚠️Common Wrong Answer
Choosing a complex model like deep learning, which may not meet latency requirements.
Step 4
Evaluating the Model
How will you evaluate the model's performance?
💡 Hint
Consider both offline and online metrics.
Key Decision
Use offline metrics like AUC and online metrics like retention rate improvement.
Interviewer Expectation
Mid: Mentions basic metrics. Senior: Explains why different metrics are needed for offline vs. online evaluation.
⚠️Common Wrong Answer
Focusing only on offline metrics, missing the real-world impact.
Step 5
Deployment Considerations
What are the key considerations for deploying this model in production?
💡 Hint
Think about integration with existing systems and maintaining low latency.
Key Decision
Ensure integration with existing systems and optimize for low latency.
Interviewer Expectation
Mid: Discusses basic deployment steps. Senior: Considers system integration and latency optimization.
⚠️Common Wrong Answer
Overlooking integration challenges, leading to deployment delays.
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Identifies limitations of rule-based systems for churn prediction.
  • Selects relevant data sources like user activity and support interactions.
  • Chooses a model that balances complexity and latency.
  • Evaluates model using both offline and online metrics.
Good to Have - 30%
  • Explains how ML models adapt to new patterns.
  • Justifies data source choices with examples.
  • Discusses trade-offs in model selection.
  • Considers system integration in deployment.
Bonus - 10%
  • Quantifies latency differences between model choices.
  • Proposes a plan for continuous model improvement.
  • Considers ethical implications of churn prediction.
✅
Model Answer
▼ Reveal

The ideal solution involves using machine learning to predict customer churn due to the complexity of user behavior. Key data sources include user activity and support interactions. A lightweight model like logistic regression is chosen to meet latency requirements. The model is evaluated using both offline metrics like AUC and online metrics like retention rate improvement. Deployment focuses on integration with existing systems and maintaining low latency.

Key Decisions
  • Machine learning is chosen over rule-based systems due to the complexity of user behavior.
  • User activity and support interactions are prioritized as key data sources.
  • A lightweight model is selected to ensure fast inference and meet latency constraints.
  • Both offline and online metrics are used to evaluate model performance.
  • Deployment considerations include system integration and latency optimization.
Anti-patterns
Proposing a rule-based system that cannot adapt to complex user behavior.
Ignoring critical data sources like support interactions.
Choosing a complex model that fails to meet latency requirements.
Focusing solely on offline metrics, missing real-world impact.

Practice

(1/5)
1. 
import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.layer_norm = nn.LayerNorm(10)

    def forward(self, x):
        return self.layer_norm(x)

model = MyModel()
input_tensor = torch.randn(5, 10)
output = model(input_tensor)
print(output.shape)
What is the output shape of the above code?
easy
A. (10, 5)
B. (5,)
C. (5, 10)
D. (10,)

Solution

  1. Step 1: Analyze the LayerNorm application.

    LayerNorm normalizes across the last dimension, maintaining the input shape.

  2. Step 2: Determine the output shape.

    The input tensor has shape (5, 10), and LayerNorm does not alter the shape, so the output is also (5, 10).

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    LayerNorm keeps input shape. [OK]
Hint: LayerNorm keeps input shape. [OK]
Common Mistakes:
2. 
from sklearn.tree import DecisionTreeClassifier
import numpy as np

X = np.array([[0, 0], [1, 1], [1, 0], [0, 1]])
y = np.array([0, 1, 1, 0])

clf = DecisionTreeClassifier(criterion='gini')
clf.fit(X, y)

print(clf.predict([[0, 0], [1, 1]]))
What is the output of the above code?
easy
A. [0, 0]
B. [1, 0]
C. [0, 1]
D. [1, 1]

Solution

  1. Step 1: Understand the training data.

    The data is perfectly separable, and the decision tree will learn to classify based on the given labels.

  2. Step 2: Predict using the trained model.

    For input [0, 0], the label is 0, and for [1, 1], the label is 1.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Check training labels for predictions. [OK]
Hint: Check training labels for predictions. [OK]
Common Mistakes:
3. Given the following Python code snippet, what will be the output? 
import numpy as np

data = np.array([1, 2, 3, 4, 5])
mask = data > 3
result = data[mask]
print(result)
easy
A. [4, 5]
B. [1, 2, 3]
C. [3, 4, 5]
D. Throws an error

Solution

  1. Step 1: Understand the code logic

    The mask is applied to filter elements greater than 3.

  2. Step 2: Determine the output

    Only elements 4 and 5 satisfy the condition, so result is [4, 5].

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Mask filters elements > 3. [OK]
Hint: Mask filters elements > 3. [OK]
Common Mistakes:
4. You are tasked with designing a system to handle label drift in a real-time recommendation engine. Which approach would be most effective?
hard
A. Implement a feedback loop to update the model with recent user interactions.
B. Use a static label mapping based on historical data.
C. Increase the size of the training dataset.
D. Apply feature selection to reduce dimensionality.

Solution

  1. Step 1: Identify the best approach for handling label drift.

    A feedback loop allows the model to adapt to changes in label distribution by learning from recent interactions.

  2. Step 2: Eliminate wrong options.

    Static mappings (Use a static label mapping based on historical data.) and increasing dataset size (Increase the size of the training dataset.) don't adapt to real-time changes. Feature selection (Apply feature selection to reduce dimensionality.) doesn't address label drift.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Feedback loop adapts to label drift [OK]
Hint: Feedback loop adapts to label drift [OK]
Common Mistakes:
5. You need to implement a real-time recommendation system. Which approach is most suitable given the need for low latency and high throughput?
hard
A. Online learning with a streaming framework
B. Offline training with periodic updates
C. Batch processing with Spark
D. Using a pre-trained model with batch inference

Solution

  1. Step 1: Identify the requirements.

    Real-time systems require low latency and high throughput.

  2. Step 2: Choose the approach that meets these requirements.

    Online learning with a streaming framework allows for real-time updates and low latency.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Streaming fits real-time needs. [OK]
Hint: Streaming fits real-time needs. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard