Bird
Raised Fist0

Implement Forward Pass and Backpropagation - NumPy Only

CODINGadvancedInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
▶
Execution Trace
Step 1/5
X

Input X is initialized with shape (3, 1) with values [[0.5], [0.2], [0.1]].

🎯 Understanding input dimensions is crucial for matrix operations.
X = np.array([[0.5], [0.2], [0.1]])
Z1

Z1 is computed as W1*X + b1, resulting in [[0.3], [0.7]].

🎯 Matrix multiplication and bias addition are key in linear transformations.
W1 = np.array([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]])
b1 = np.array([[0.1], [0.2]])
Z1 = np.dot(W1, X) + b1
A1

A1 is computed using ReLU activation, resulting in [[0.3], [0.7]].

🎯 ReLU activation helps introduce non-linearity.
A1 = np.maximum(0, Z1)
Z2

Z2 is computed as W2*A1 + b2, resulting in [[1.2]].

🎯 Output layer transformation combines hidden activations.
W2 = np.array([[0.7, 0.8]])
b2 = np.array([[0.3]])
Z2 = np.dot(W2, A1) + b2
A2

A2 is computed using Softmax, resulting in [[1.0]].

🎯 Softmax converts logits to probabilities.
exp_Z2 = np.exp(Z2 - np.max(Z2))
A2 = exp_Z2 / np.sum(exp_Z2)
📊
Diagram
Diagram
┌──────────────────────────────────────────────────────────────┐
│ Input Layer (X)                                              │
│ ┌───────────┐                                                │
│ │  X[0]     │                                                │
│ └───────────┘                                                │
│     │                                                        │
│     ▼                                                        │
│ Hidden Layer (Z1 = W1*X + b1)                                │
│ ┌───────────┐                                                │
│ │  Z1[0]    │                                                │
│ └───────────┘                                                │
│     │                                                        │
│     ▼                                                        │
│ Activation (A1 = ReLU(Z1))                                   │
│ ┌───────────┐                                                │
│ │  A1[0]    │                                                │
│ └───────────┘                                                │
│     │                                                        │
│     ▼                                                        │
│ Output Layer (Z2 = W2*A1 + b2)                               │
│ ┌───────────┐                                                │
│ │  Z2[0]    │                                                │
│ └───────────┘                                                │
│     │                                                        │
│     ▼                                                        │
│ Activation (A2 = Softmax(Z2))                                │
│ ┌───────────┐                                                │
│ │  A2[0]    │                                                │
│ └───────────┘                                                │
└──────────────────────────────────────────────────────────────┘
Labels
  • Input Layer (X): Initial data fed into the network
  • Hidden Layer (Z1): Linear transformation of input
  • Activation (A1): Non-linear transformation using ReLU
  • Output Layer (Z2): Linear transformation of hidden layer
  • Activation (A2): Final output using Softmax
🔑
Key Moments
▲
Q: Why is ReLU used as the activation function in the hidden layer?
A: ReLU introduces non-linearity, allowing the network to learn complex patterns.
Q: What role does the Softmax function play in the output layer?
A: Softmax converts the output logits into probabilities, making them interpretable as class predictions.
Q: How does the bias term affect the linear transformation in each layer?
A: The bias term shifts the activation function, allowing the model to fit the data better.
Trace Summary

This visual trace document explains the forward pass and backpropagation process in a neural network using NumPy. The diagram shows the flow of data through the network, highlighting key transformations at each layer. The trace steps detail the computation of weighted sums, application of activation functions, and the final output probabilities. Key moments focus on understanding the role of activation functions like ReLU and Softmax, and the importance of bias terms in linear transformations. This document provides a comprehensive understanding of how neural networks process data and learn from it.

Primary Focus

✓ Deep Learning Coding with NumPy

Practice

(1/5)
1. 
import numpy as np

data = np.array([1, 2, 3, 4, 5])
mean = np.mean(data)
std = np.std(data)

standardized_data = (data - mean) / std
print(standardized_data)
What does the code output?
easy
A. An array with values [0, 1, 2, 3, 4]
B. An array with values [0, 0.5, 1, 1.5, 2]
C. An array with values [-1.41, -0.71, 0, 0.71, 1.41]
D. An array with values [-2, -1, 0, 1, 2]

Solution

  1. Step 1: Understand Standardization

    Standardization transforms data to have a mean of 0 and a standard deviation of 1.

  2. Step 2: Calculate the Output

    The standardized values are calculated as (data - mean) / std, resulting in [-1.41, -0.71, 0, 0.71, 1.41].

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Standardization centers data around 0. [OK]
Hint: Standardization centers data around 0. [OK]
Common Mistakes:
2. What does the following Python code snippet output when implementing a simple bagging technique for a Random Forest?
import numpy as np
from sklearn.utils import resample

X = np.array([[1, 2], [3, 4], [5, 6], [7, 8]])

# Bagging
bagged_X = resample(X, n_samples=3, replace=True, random_state=42)
print(bagged_X)
easy
A. An array with 3 samples, possibly with duplicates, from X
B. An array with 3 unique samples from X
C. An array with 4 samples, each appearing once
D. An array with 2 samples, each appearing twice

Solution

  1. Step 1: Understand Bagging with Replacement

    Bagging involves sampling with replacement, meaning samples can be repeated.

  2. Step 2: Analyze Code Output

    The code uses resample with replace=True, so duplicates are possible.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Bagging allows duplicates [OK]
Hint: Bagging allows duplicates [OK]
Common Mistakes:
3. A Gaussian Naive Bayes model is giving poor performance on a dataset. Which of the following is the most likely cause?
medium
A. The model has too many parameters.
B. The features are not normally distributed.
C. The dataset is too small.
D. The features are highly correlated.

Solution

  1. Step 1: Identify Gaussian Naive Bayes assumptions

    Gaussian Naive Bayes assumes that features are normally distributed.

  2. Step 2: Evaluate each option

    The features are not normally distributed. is correct because non-normal distribution violates the model's assumptions. The model has too many parameters. is incorrect as GaussianNB is simple. The dataset is too small. is less likely as small datasets affect all models. The features are highly correlated. is incorrect as GaussianNB can handle correlated features.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    GaussianNB assumes normal distribution. [OK]
Hint: GaussianNB assumes normal distribution. [OK]
Common Mistakes:
4. How can you optimize the following data augmentation pipeline for memory efficiency? 
def augment_image(img):
    img = np.flip(img, axis=1)
    img = img[10:110, 10:110]
    img = (img - np.mean(img)) / np.std(img)
    return img
hard
A. Chain operations without intermediate assignments.
B. Pre-compute mean and std for normalization.
C. Use a smaller dtype for the image array.
D. Use in-place operations for flipping and cropping.

Solution

  1. Step 1: Identify memory inefficiencies

    Intermediate assignments create unnecessary copies.

  2. Step 2: Evaluate optimization strategies

    Chaining operations reduces memory usage by avoiding intermediate arrays.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Chain operations to reduce memory usage. [OK]
Hint: Chain operations to reduce memory usage. [OK]
Common Mistakes:
5. How can you optimize the memory usage of the Adam optimizer in a large-scale deep learning model?
hard
A. Reduce the batch size to decrease memory footprint.
B. Use half-precision floating points for parameter storage.
C. Share momentum and velocity arrays across layers.
D. Store only the last two gradients instead of all past gradients.

Solution

  1. Step 1: Identify memory-intensive components

    Adam requires storing additional momentum and velocity arrays, which can be memory-intensive.

  2. Step 2: Optimize storage precision

    Using half-precision floating points reduces memory usage significantly while maintaining sufficient precision for most deep learning tasks.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Use half-precision to save memory in Adam [OK]
Hint: Use half-precision to save memory in Adam [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard