Minimizing multivariate functions (minimize) in SciPy

Introduction

We use minimization to find the lowest point of a function with many variables. This helps us solve problems like finding the best settings or lowest cost.

Finding the best combination of ingredients to minimize cost in a recipe.

Adjusting parameters in a machine learning model to reduce error.

Optimizing investment portfolios to minimize risk.

Tuning settings in a manufacturing process to reduce waste.

Finding the shortest path or least energy in physics problems.

Syntax

SciPy

from scipy.optimize import minimize

result = minimize(fun, x0, method=None, jac=None, options=None)

# fun: function to minimize
# x0: starting guess (array-like)
# method: optimization algorithm (optional)
# jac: gradient function (optional)
# options: dictionary of solver options (optional)

The fun function should take an array and return a single number.

The x0 is your initial guess for the variables.

Examples

Minimize a simple function starting from (0,0).

SciPy

from scipy.optimize import minimize

def f(x):
    return (x[0]-1)**2 + (x[1]-2.5)**2

result = minimize(f, [0, 0])
print(result.x)

Use gradient information with BFGS method for faster convergence.

SciPy

from scipy.optimize import minimize

def f(x):
    return (x[0]-1)**2 + (x[1]-2.5)**2

def grad(x):
    return [2*(x[0]-1), 2*(x[1]-2.5)]

result = minimize(f, [0, 0], jac=grad, method='BFGS')
print(result.x)

Sample Program

This program finds the minimum of a simple function with two variables. It starts guessing at (0,0) and finds the point where the function is smallest.

SciPy

from scipy.optimize import minimize

def objective(x):
    # A function with two variables
    return (x[0] - 3)**2 + (x[1] + 1)**2 + 5

# Starting guess
x0 = [0, 0]

# Run minimization
result = minimize(objective, x0)

print('Minimum value found:', result.fun)
print('At position:', result.x)

Important Notes

Choosing a good starting point x0 can help the solver find the minimum faster.

Some methods need the gradient (jacobian) to work well, but it is optional.

The result object contains useful info like result.x (solution) and result.fun (minimum value).

Summary

Use scipy.optimize.minimize to find the lowest point of functions with many variables.

Provide the function and a starting guess to begin the search.

Check the result for the minimum value and where it happens.