Practice
1. In a deep learning model, when is it more appropriate to use Layer Normalization instead of Batch Normalization?
easy
Solution
Step 1: Understand the context of normalization techniques.
Batch Normalization relies on batch statistics, which can be unstable with small batch sizes. Layer Normalization uses layer statistics, making it more stable for small batches or online learning.Step 2: Eliminate wrong options.
When the model is trained on a very large batch size. is incorrect because large batch sizes are suitable for Batch Normalization. When the model has a very deep architecture. is incorrect because depth doesn't directly affect the choice between these normalizations. When the model is trained on a balanced dataset. is irrelevant to normalization choice.Final Answer:
Option B -> Option BQuick Check:
Layer Norm for small batches or online learning. [OK]
Hint: Layer Norm for small batches or online learning. [OK]
Common Mistakes:
2. Given the following Python code snippet using a pre-trained YOLO model, what will be the output shape of the detection results?
import torch
from torchvision.models.detection import yolo_v3
model = yolo_v3(pretrained=True)
model.eval()
input_tensor = torch.rand(1, 3, 416, 416)
output = model(input_tensor)
print(output[0]['boxes'].shape)easy
Solution
Step 1: Understand YOLO output
YOLO outputs a list of detections, each with a bounding box of 4 coordinates.Step 2: Determine output shape
The shape is (N, 4) where N is the number of detected objects.Final Answer:
Option B -> Option BQuick Check:
YOLO boxes = (N, 4) [OK]
Hint: YOLO boxes = (N, 4) [OK]
Common Mistakes:
3. Given the following PyTorch code snippet for an LSTM layer, what is the shape of the output
h_n?
import torch
lstm = torch.nn.LSTM(input_size=10, hidden_size=20, num_layers=2)
input = torch.randn(5, 3, 10)
output, (h_n, c_n) = lstm(input)easy
Solution
Step 1: Understand LSTM output shapes.
Theh_nshape is (num_layers, batch_size, hidden_size).Step 2: Apply the given parameters.
Here, num_layers=2, batch_size=3, hidden_size=20, so the shape is (2, 3, 20).Final Answer:
Option B -> Option BQuick Check:
h_n shape: (layers, batch, hidden). [OK]
Hint: h_n shape: (layers, batch, hidden). [OK]
Common Mistakes:
4. In a scenario where computational resources are limited, which approach is more suitable for sequence-to-sequence tasks and why?
medium
Solution
Step 1: Evaluate computational efficiency.
Attention mechanisms allow for parallel processing, making them more efficient with limited resources.Step 2: Eliminate wrong options.
Use RNNs because they require less memory than attention mechanisms. is incorrect as attention mechanisms can be more memory efficient due to parallelism. Use RNNs because they are faster to train than attention mechanisms. is incorrect because RNNs are slower due to sequential processing. Use attention mechanisms because they are more interpretable than RNNs. is incorrect as interpretability is not the main concern here.Final Answer:
Option C -> Option CQuick Check:
Parallel processing saves resources. [OK]
Hint: Parallel processing saves resources. [OK]
Common Mistakes:
5. A neural network is not learning effectively, and the gradients are extremely small. What is the most likely cause?
medium
Solution
Step 1: Identify symptoms of vanishing gradients
Extremely small gradients often indicate the vanishing gradient problem, typically caused by certain activation functions like sigmoid or tanh.Step 2: Eliminate wrong options
The learning rate is too high. is incorrect because a high learning rate would cause divergence. The network is too shallow. is incorrect because a shallow network is less prone to vanishing gradients. The dataset is too small. is unrelated to gradient size.Final Answer:
Option C -> Option CQuick Check:
Small gradients often mean vanishing issue. [OK]
Hint: Small gradients often mean vanishing issue. [OK]
Common Mistakes: