Practice
1. In the context of recurrent neural networks, which mechanism is specifically designed to address the vanishing gradient problem?
easy
Solution
Step 1: Identify the mechanism for vanishing gradient.
LSTMs are specifically designed to handle vanishing gradients by using gating mechanisms.Step 2: Eliminate wrong options.
Dropout (A) is for regularization, Batch Normalization (B) stabilizes learning, and Weight Decay (D) is for preventing overfitting.Final Answer:
Option A -> Option AQuick Check:
LSTM handles vanishing gradients with gates. [check]
Hint: LSTM handles vanishing gradients with gates. [check]
Common Mistakes:
2. You notice that your LSTM model is not learning effectively and suspect vanishing gradients. What is a likely cause specific to LSTM architecture?
medium
Solution
Step 1: Identify LSTM-specific issues.
Incorrect initialization of forget gate biases can lead to vanishing gradients.Step 2: Eliminate wrong options.
Too many layers can cause vanishing gradients, but it's not LSTM-specific. High learning rates cause exploding gradients, and dropout affects overfitting, not gradient flow.Final Answer:
Option A -> Option AQuick Check:
Forget gate bias affects gradient flow. [OK]
Hint: Forget gate bias affects gradient flow. [OK]
Common Mistakes:
3. In a scenario where computational resources are limited, which optimizer would you choose for faster convergence and why?
medium
Solution
Step 1: Consider optimizer efficiency.
Adam combines the benefits of both AdaGrad and RMSProp, leading to faster convergence.Step 2: Evaluate resource constraints.
Adam is computationally efficient and converges faster than SGD with momentum and AdaGrad.Final Answer:
Option A -> Option AQuick Check:
Adam is efficient for limited resources. [check]
Hint: Adam is efficient for limited resources. [check]
Common Mistakes:
4. In a real-time system where predictions need to be made quickly, which approach to linear regression is more suitable and why?
hard
Solution
Step 1: Consider real-time constraints
Stochastic Gradient Descent (SGD) updates weights after each data point, allowing for faster, more frequent updates suitable for real-time systems.Step 2: Eliminate wrong options
Batch Gradient Descent (A) is too slow for real-time, Least Squares (C) is not iterative, and Mini-batch (D) is a compromise but not as fast as SGD.Final Answer:
Option A -> Option AQuick Check:
SGD updates weights after each data point. [OK]
Hint: SGD updates weights after each data point. [OK]
Common Mistakes:
5. You are tasked with designing a recommendation system for a high-dimensional dataset. Which approach would best handle the curse of dimensionality while maintaining performance?
hard
Solution
Step 1: Identify the challenge of high dimensions.
Matrix factorization reduces dimensionality effectively.Step 2: Eliminate wrong options.
Implement a simple content-based filtering system. doesn't reduce dimensions. Apply a deep learning model with many hidden layers. may overfit due to high dimensions. Use a rule-based system to manually curate recommendations. is not scalable for large datasets.Final Answer:
Option B -> Option BQuick Check:
Matrix factorization reduces dimensions, aids recommendations. [OK]
Hint: Matrix factorization reduces dimensions, aids recommendations. [OK]
Common Mistakes:
