Optimization callbacks and monitoring in SciPy - Step-by-Step Execution

Concept Flow - Optimization callbacks and monitoring

Start Optimization

↓

Iteration begins

↓

Evaluate function

↓

Call callback with current state

↓

Check stopping criteria

No Yes↓

Update variables

↓

Next iteration

The optimization runs in steps. After each step, it evaluates the function, calls the callback to monitor progress, then decides to continue or stop.

Execution Sample

SciPy

from scipy.optimize import minimize

def callback(xk):
    print(f"Current x: {xk}")

res = minimize(lambda x: (x-3)**2, x0=0, callback=callback)

This code runs an optimization to find x that minimizes (x-3)^2, printing x at each step.

Execution Table

Step	Current x	Function Value	Callback Output	Action
1	[0.0]	9.0	Current x: [0.0]	Update x towards 3
2	[1.8]	1.44	Current x: [1.8]	Update x towards 3
3	[2.52]	0.2304	Current x: [2.52]	Update x towards 3
4	[2.808]	0.0467	Current x: [2.808]	Update x towards 3
5	[2.965]	0.0012	Current x: [2.965]	Update x towards 3
6	[2.999]	0.0000	Current x: [2.999]	Converged, stop
7	[3.0]	0.0	Current x: [3.0]	Optimization ends

💡 Optimization stops when x is close enough to 3 and function value is near zero.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	After 5	After 6	Final
x	0.0	1.8	2.52	2.808	2.965	2.999	3.0	3.0
function_value	9.0	1.44	0.2304	0.0467	0.0012	0.0000	0.0	0.0

Key Moments - 2 Insights

Why does the callback print the current x before the variable updates?

The callback is called after the function evaluation at the current x but before the next update, as shown in execution_table rows where callback output matches current x before action.

Why does the optimization stop even though x is not exactly 3?

Optimization stops when the function value is close enough to zero (converged), not necessarily exactly at 3, as seen in step 6 where function value is near zero.

Visual Quiz - 3 Questions

Look at the execution table, what is the function value at step 3?

A 0.2304

B 1.44

C 0.0467

D 9.0

💡 Hint

Check the 'Function Value' column at step 3 in the execution_table.

At which step does the optimization decide to stop?

A Step 5

B Step 7

C Step 6

D Step 4

💡 Hint

Look at the 'Action' column where it says 'Converged, stop'.

If the callback was removed, what would change in the execution table?

A Function values would be different

B No callback output column and no printed current x values

C Optimization would not stop

D x values would not update

💡 Hint

Callback only monitors progress; it does not affect variable updates or stopping.

Concept Snapshot

Optimization callbacks let you monitor progress each step.
Callback runs after function evaluation but before variable update.
Use callback to print or log current variables.
Optimization stops when criteria met, not necessarily exact solution.
Callbacks do not change optimization behavior, only observe it.

Full Transcript

In optimization with scipy, the process starts by evaluating the function at an initial guess. After each evaluation, a callback function is called with the current variables. This callback can print or log the current state to monitor progress. Then the optimizer updates the variables to move closer to the minimum. This repeats until the stopping criteria are met, such as the function value being close enough to zero. The callback helps track the optimization without changing its behavior. The example shows x moving from 0 to 3, with the callback printing x at each step. The optimization stops when x is near 3 and the function value is near zero.