Backpropagation concept in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of backpropagation in training a neural network?

Imagine you are teaching a robot to recognize cats in photos. The robot guesses, but sometimes it is wrong. How does backpropagation help the robot improve its guesses?

A It randomly changes the robot's settings without considering the errors.

B It changes the robot's hardware to make it faster at recognizing cats.

C It adjusts the robot's internal settings by calculating how wrong the guess was and changing the settings to reduce future errors.

D It collects more photos of cats to show the robot for better learning.

💡 Hint

Think about how the robot learns from mistakes by adjusting what it knows.

Backpropagation calculates the error between the robot's guess and the correct answer, then adjusts the internal settings (weights) to reduce this error in future guesses.

❓ Predict Output

intermediate

What is the output of the gradient calculation step in backpropagation?

Given a simple neural network with one weight w and loss function L = (w * x - y)^2, where x=2 and y=4, what is the gradient of L with respect to w when w=1?

ML Python

x = 2
y = 4
w = 1
L = (w * x - y)**2
# Calculate dL/dw
dL_dw = 2 * (w * x - y) * x
print(dL_dw)

A -4

B 8

C 4

D -8

💡 Hint

Use the chain rule: dL/dw = 2 * (w*x - y) * x

Calculate error: (1*2 - 4) = -2. Then gradient: 2 * (-2) * 2 = -8.

❓ Model Choice

advanced

Which neural network architecture benefits most from backpropagation through time (BPTT)?

Backpropagation through time is a special version of backpropagation used for certain models. Which model type uses BPTT to learn from sequences?

A Recurrent Neural Networks (RNNs) for language modeling

B Feedforward Neural Networks for tabular data

C Convolutional Neural Networks (CNNs) for image recognition

D Autoencoders for dimensionality reduction

💡 Hint

Think about models that process data step-by-step over time.

BPTT is used to train RNNs because they handle sequences and need to propagate errors back through time steps.

❓ Hyperparameter

advanced

Which hyperparameter directly controls the size of weight updates during backpropagation?

When training a neural network, which setting decides how big each step is when adjusting weights after calculating gradients?

A Learning rate

B Batch size

C Number of epochs

D Activation function

💡 Hint

It is a small number that multiplies the gradient to update weights.

The learning rate scales the gradient during weight updates, controlling how fast or slow the model learns.

❓ Metrics

expert

After training a neural network with backpropagation, which metric best indicates if the model is overfitting?

You trained a model and see that training loss keeps decreasing but validation loss starts increasing. Which metric behavior shows overfitting?

A Both training and validation accuracy increase steadily

B Training accuracy increases while validation accuracy decreases

C Training loss increases while validation loss decreases

D Both training and validation loss decrease steadily

💡 Hint

Overfitting means the model learns training data too well but fails on new data.

When training accuracy improves but validation accuracy worsens, the model is overfitting.