Gradient descent optimization in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What does the learning rate control in gradient descent?

Imagine you are walking down a hill to reach the lowest point. In gradient descent, what does the learning rate control?

A The size of each step you take down the hill

B The speed at which the hill moves

C The total distance from the hill to the valley

D The direction you walk towards the hilltop

💡 Hint

Think about how big or small your steps are when walking down.

The learning rate controls how big each step is when moving towards the minimum. Too big steps can overshoot, too small steps slow progress.

❓ Predict Output

intermediate

Output of gradient descent update step

What is the value of w after one gradient descent update?

ML Python

w = 2.0
learning_rate = 0.1
gradient = 3.0
w = w - learning_rate * gradient
print(round(w, 2))

A 1.3

B 1.7

C 2.3

D 2.0

💡 Hint

Use the formula: new_w = old_w - learning_rate * gradient

Calculate: 2.0 - 0.1 * 3.0 = 2.0 - 0.3 = 1.7

❓ Hyperparameter

advanced

Choosing the right learning rate for convergence

Which learning rate is most likely to cause the gradient descent to converge smoothly to the minimum?

A 10.0

B 1.0

C 0.0001

D 0.1

💡 Hint

Too small means slow progress, too large means overshooting.

A learning rate of 0.1 is balanced: not too small to be slow, not too large to overshoot. 1.0 or 10.0 are too big and cause divergence. 0.0001 is very slow.

❓ Metrics

advanced

Effect of learning rate on training loss curve

Which training loss curve shape best represents a too-large learning rate during gradient descent?

A Loss oscillates up and down without settling

B Loss steadily decreases and flattens near zero

C Loss increases steadily over time

D Loss remains constant throughout training

💡 Hint

Think about what happens if steps are too big and jump around the minimum.

A too-large learning rate causes the loss to jump around the minimum, causing oscillations instead of smooth decrease.

🔧 Debug

expert

Identify the error in this gradient descent update code

What error will this code produce when running?

ML Python

w = 1.0
learning_rate = 0.05
gradient = None
w = w - learning_rate * gradient
print(w)

A SyntaxError: invalid syntax

B NameError: name 'gradient' is not defined

C TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'

D No error, prints 1.0

💡 Hint

Check the type of gradient before multiplication.

Multiplying a float by None causes a TypeError because None is not a number.