Practice
1.
import torch
import torch.nn.functional as F
query = torch.tensor([[1.0, 0.0]])
key = torch.tensor([[0.5, 0.5], [1.0, 0.0]])
value = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
attention_scores = torch.matmul(query, key.T)
attention_weights = F.softmax(attention_scores, dim=-1)
output = torch.matmul(attention_weights, value)
print(output)easy
Solution
Step 1: Calculate attention scores.
Query-key dot product gives scores: [[0.5, 1.0]].Step 2: Apply softmax to scores.
Softmax gives weights: [[0.2689, 0.7311]].Step 3: Compute weighted sum of values.
Output = [[0.2689*1.0 + 0.7311*3.0, 0.2689*2.0 + 0.7311*4.0]] = [[1.5, 3.0]].Final Answer:
Option A -> Option AQuick Check:
Softmax weights applied to values. [OK]
Hint: Softmax weights applied to values. [OK]
Common Mistakes:
2.
import numpy as np
x = np.array([1, 2, 3])
y = np.array([4, 5, 6])
result = np.dot(x, y)
print(result)easy
Solution
Step 1: Understand np.dot behavior
np.dot for 1D arrays computes the dot product, which is a scalar.Step 2: Calculate the dot product
1*4 + 2*5 + 3*6 = 32.Final Answer:
Option C -> Option CQuick Check:
np.dot 1D arrays = scalar [OK]
Hint: np.dot 1D arrays = scalar [OK]
Common Mistakes:
3. In the context of deep learning, why might a deep network be preferred over a shallow network for complex tasks?
easy
Solution
Step 1: Understand the advantage of deep networks
Deep networks can capture hierarchical features, which is crucial for complex tasks.Step 2: Eliminate wrong options
Deep networks can represent any function with fewer parameters. is incorrect because deep networks do not always use fewer parameters. Shallow networks are always faster to train. is incorrect as shallow networks are not always faster. Shallow networks generalize better on all tasks. is incorrect because shallow networks do not always generalize better.Final Answer:
Option B -> Option BQuick Check:
Deep networks capture hierarchies. [OK]
Hint: Deep networks capture hierarchies. [OK]
Common Mistakes:
4. When designing a CNN for image classification, why might you choose to use a smaller receptive field in the initial layers?
medium
Solution
Step 1: Understand receptive fields
Smaller receptive fields in initial layers help capture fine-grained local features, which are crucial for understanding detailed patterns.Step 2: Eliminate wrong options
To capture more global features early on is incorrect because global features are captured in deeper layers. To reduce the computational cost of the network is incorrect as smaller receptive fields don't necessarily reduce computational cost. To increase the depth of the network without increasing parameters is incorrect because depth and parameters are not directly related to receptive field size.Final Answer:
Option C -> Option CQuick Check:
Smaller fields capture local details. [OK]
Hint: Smaller fields capture local details. [OK]
Common Mistakes:
5. How would you adapt a Random Forest model to improve its robustness to adversarial attacks?
hard
Solution
Step 1: Understand adversarial robustness
Adversarial training involves training the model with perturbed data to improve robustness.Step 2: Evaluate adaptation strategies
Use adversarial training with perturbed data is correct because adversarial training directly addresses robustness. Increase the number of trees significantly is incorrect as more trees don't inherently improve adversarial robustness. Reduce the depth of each tree is incorrect because reducing depth may not address adversarial examples. Increase the max_features parameter is incorrect as increasing max_features doesn't target adversarial robustness.Final Answer:
Option A -> Option AQuick Check:
Adversarial training improves robustness. [OK]
Hint: Adversarial training improves robustness. [OK]
Common Mistakes: