Optimization callbacks and monitoring in SciPy

Introduction

Callbacks help you watch and control the progress of an optimization. They let you see how the solution improves step-by-step.

You want to see how close the optimizer is to the best answer during the run.

You need to stop the optimization early if it is taking too long or not improving.

You want to log or print the current solution at each step for debugging.

You want to update a graph or progress bar while the optimizer runs.

Syntax

SciPy

def callback(xk):
    # xk is the current solution
    # Add your monitoring or stopping logic here
    pass

result = scipy.optimize.minimize(fun, x0, callback=callback)

The callback function is called after each iteration with the current solution.

You can use callback to print, log, or stop the optimization.

Examples

This prints the current solution at each step.

SciPy

def callback(xk):
    print(f"Current solution: {xk}")

This example shows how you might stop the optimization early based on a condition.

SciPy

def callback(xk):
    if some_condition(xk):
        print("Stopping early")
        return True  # Some optimizers support this to stop early

This saves each step's solution to a list for later analysis.

SciPy

def callback(xk):
    history.append(xk.copy())

Sample Program

This program minimizes a simple function. It prints and saves each step's solution using a callback.

SciPy

import numpy as np
from scipy.optimize import minimize

# Objective function: simple quadratic
fun = lambda x: (x[0]-3)**2 + (x[1]+1)**2

history = []

def callback(xk):
    print(f"Step solution: {xk}")
    history.append(xk.copy())

x0 = np.array([0, 0])
result = minimize(fun, x0, callback=callback)

print(f"Final solution: {result.x}")

Important Notes

Not all optimizers support stopping early via callback return values.

Callbacks run often, so keep them fast to avoid slowing optimization.

Use callbacks to collect data for graphs or logs to understand optimization behavior.

Summary

Callbacks let you watch optimization progress step-by-step.

You can print, log, or stop optimization using callbacks.

Callbacks help you understand and control how optimization runs.