Practice
1. What does the following Python code output when implementing logistic regression with a sigmoid activation function?
import numpy as np
def sigmoid(z):
return 1 / (1 + np.exp(-z))
z = np.array([0, 2, -2])
output = sigmoid(z)
print(output)easy
Solution
Step 1: Calculate sigmoid for each element
Sigmoid function is applied element-wise:sigmoid(0) = 0.5,sigmoid(2) ≈ 0.88,sigmoid(-2) ≈ 0.12.Step 2: Check precision of calculations
Common mistake is rounding errors. Correct values are[0.5, 0.880797, 0.119203], rounded to[0.5, 0.88, 0.12].Final Answer:
Option C -> Option CQuick Check:
Sigmoid(0)=0.5, Sigmoid(2)≈0.88, Sigmoid(-2)≈0.12 [OK]
Hint: Sigmoid(0)=0.5, Sigmoid(2)≈0.88, Sigmoid(-2)≈0.12 [OK]
Common Mistakes:
2. What is the time complexity of a single update step in Mini-Batch SGD with Momentum for a model with
n parameters?easy
Solution
Step 1: Identify operations involved
Each update step involves computing the gradient and updating the velocity and parameters, all of which are linear operations with respect to the number of parameters.Step 2: Determine the complexity
Since each operation is linear, the overall complexity isO(n).Final Answer:
Option A -> Option AQuick Check:
Each parameter update is linear in complexity [OK]
Hint: Each parameter update is linear in complexity [OK]
Common Mistakes:
3. In a decision tree, how would you handle a dataset with continuous features?
medium
Solution
Step 1: Consider handling continuous features
Discretizing into bins allows decision trees to handle continuous data effectively.Step 2: Eliminate wrong options
One-hot encoding (A) is for categorical data; ignoring (C) loses information; using directly (D) can lead to overfitting.Final Answer:
Option C -> Option CQuick Check:
Continuous? Discretize into bins [OK]
Hint: Continuous? Discretize into bins [OK]
Common Mistakes:
4. In the context of implementing linear regression with gradient descent, how would you handle a dataset with missing values?
medium
Solution
Step 1: Consider impact of missing values
Ignoring or zero-filling can bias results; removing samples reduces data.Step 2: Choose appropriate imputation
Mean imputation is a common strategy that maintains dataset size and reduces bias.Final Answer:
Option A -> Option AQuick Check:
Mean imputation balances bias and data retention. [OK]
Hint: Mean imputation balances bias and data retention. [OK]
Common Mistakes:
5. In the following learned positional encoding implementation, the model is not learning as expected. What is the most likely cause?
import torch
import torch.nn as nn
class LearnedPositionalEncoding(nn.Module):
def __init__(self, max_len, d_model):
super().__init__()
self.positional_encoding = nn.Parameter(torch.zeros(max_len, d_model))
def forward(self, x):
return x + self.positional_encoding[:x.size(1), :]
model = LearnedPositionalEncoding(100, 512)
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)medium
Solution
Step 1: Identify the initialization issue
The positional encoding is initialized to zeros, which may not provide a useful starting point for learning.Step 2: Eliminate wrong options
The learning rate is too low. is incorrect because the learning rate is standard. The optimizer is not suitable for this task. is incorrect because Adam is generally suitable. The positional encoding is not being updated. is incorrect because the parameter is part of the model and should be updated.Final Answer:
Option A -> Option AQuick Check:
Check initialization of learned parameters. [OK]
Hint: Check initialization of learned parameters. [OK]
Common Mistakes: