Practice
1. In the context of A/B testing for ML models, which method is best suited for ensuring that temporal trends do not bias the results?
easy
Solution
Step 1: Understand the methods
Interleaving method allows for simultaneous comparison, reducing temporal bias.Step 2: Eliminate wrong options
Holdout and cross-validation do not address temporal trends. Causal inference is more about understanding causality than handling temporal trends.Final Answer:
Option B -> Option BQuick Check:
Interleaving reduces temporal bias. [OK]
Hint: Interleaving reduces temporal bias. [OK]
Common Mistakes:
2. Given the following Python code snippet, what will be the output if the dataset contains data leakage?
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = LogisticRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print(accuracy_score(y_test, y_pred))easy
Solution
Step 1: Understand the impact of data leakage
Data leakage often results in overly optimistic performance metrics.Step 2: Analyze the code behavior
The code will execute without errors, but if leakage exists, the model will perform better than it should.Final Answer:
Option D -> Option DQuick Check:
Leakage inflates accuracy [OK]
Hint: Leakage inflates accuracy [OK]
Common Mistakes:
3.
import torch
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
for epoch in range(10):
for data, target in dataloader:
optimizer.zero_grad()
output = model(data)
loss = loss_fn(output, target)
loss.backward()
optimizer.step()
print(optimizer.param_groups[0]['lr'])easy
Solution
Step 1: Understand the code.
The learning rate is set to 0.1 and not changed in the loop.Step 2: Check for learning rate updates.
SGD without a scheduler keeps the learning rate constant.Final Answer:
Option C -> Option CQuick Check:
SGD without scheduler keeps lr constant. [check]
Hint: SGD without scheduler keeps lr constant. [check]
Common Mistakes:
4.
import numpy as np
from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeClassifier
X = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])
y = np.array([0, 1, 0, 1])
model = DecisionTreeClassifier()
scores = cross_val_score(model, X, y, cv=2)
print(scores)easy
Solution
Step 1: Understand cross-validation output
Cross-validation with cv=2 splits the data into two folds, returning an array of two scores.Step 2: Eliminate wrong options
An error because the dataset is too small for cross-validation. is incorrect because the code runs without error. A single accuracy score for the entire dataset. is incorrect because cross-validation returns multiple scores. An array of four accuracy scores, one for each data point. is incorrect because the number of scores equals the number of folds, not data points.Final Answer:
Option A -> Option AQuick Check:
Cross-validation returns scores per fold. [OK]
Hint: Cross-validation returns scores per fold. [OK]
Common Mistakes:
5. When deciding between Few-Shot and Chain-of-Thought prompting, which scenario is best suited for Chain-of-Thought?
medium
Solution
Step 1: Understand Chain-of-Thought
Chain-of-Thought is best for tasks requiring complex reasoning.Step 2: Eliminate wrong options
When the task involves simple factual recall. is incorrect as factual recall doesn't need reasoning. When the model needs to be fine-tuned with a few examples. is about Few-Shot, not Chain-of-Thought. When the task requires generating creative content. is unrelated to reasoning.Final Answer:
Option B -> Option BQuick Check:
Use Chain-of-Thought for complex reasoning tasks. [OK]
Hint: Use Chain-of-Thought for complex reasoning tasks. [OK]
Common Mistakes: