Optimization callbacks and monitoring in SciPy - Mini Project: Build & Apply

Optimization Callbacks and Monitoring with SciPy

📖 Scenario: You are working on a simple optimization problem where you want to find the minimum of a mathematical function. To understand how the optimization progresses, you want to monitor the values at each step using a callback function.

🎯 Goal: Build a Python script that uses SciPy's minimize function to find the minimum of a quadratic function. You will create a callback function to monitor and store the values of the variable during optimization, then print the collected values.

📋 What You'll Learn

Create a quadratic function f(x) = (x-3)^2 + 4

Create a list called history to store values of x during optimization

Define a callback function store_history that appends the current x value to history

Use scipy.optimize.minimize with method 'BFGS' to minimize f starting from x=0

Pass the callback function to minimize to monitor optimization

Print the history list after optimization completes

💡 Why This Matters

🌍 Real World

Monitoring optimization progress helps in tuning algorithms and understanding how solutions improve step-by-step in machine learning and engineering problems.

💼 Career

Knowing how to use callbacks in optimization is useful for roles in data science, machine learning engineering, and scientific computing where optimization is common.

1

Create the quadratic function

Create a function called f that takes a variable x and returns the value of the quadratic function (x - 3)**2 + 4.

SciPy

# Define the quadratic function f(x)
# Your code here

Need a hint?

Use def to define the function and return the expression (x - 3)**2 + 4.

2

Create a list to store optimization history

Create an empty list called history that will store the values of x during optimization.

SciPy

def f(x):
    return (x - 3)**2 + 4

# Create an empty list called history
# Your code here

Need a hint?

Use history = [] to create an empty list.

3

Define the callback function to store values

Define a function called store_history that takes a parameter xk and appends xk to the history list.

SciPy

def f(x):
    return (x - 3)**2 + 4

history = []

# Define callback function store_history(xk)
# Your code here

Need a hint?

Define a function with def store_history(xk): and use history.append(xk) inside it.

4

Run optimization and print history

Import minimize from scipy.optimize. Use minimize to minimize the function f starting from x0=[0] with method 'BFGS'. Pass the callback function store_history to monitor optimization. Finally, print the history list.

SciPy

def f(x):
    return (x - 3)**2 + 4

history = []

def store_history(xk):
    history.append(xk)

# Import minimize and run optimization with callback
# Print the history list
# Your code here

Need a hint?

Use from scipy.optimize import minimize. Call minimize(f, x0=[0], method='BFGS', callback=store_history). Then print history.