Method selection (Nelder-Mead, BFGS, Powell) in SciPy

Introduction

Choosing the right method helps find the best answer faster when solving math problems with computers.

You want to find the lowest point of a hill (minimum) but don't have a formula for the slope.

You have a smooth hill and can calculate the slope to guide your search.

You want a method that works well even if the hill shape is tricky or not smooth.

Syntax

SciPy

scipy.optimize.minimize(fun, x0, method='method_name')

fun is the function to minimize.

x0 is the starting guess.

Examples

Uses a simple search that does not need slope information.

SciPy

scipy.optimize.minimize(fun, x0, method='Nelder-Mead')

Uses slope info to find the minimum faster on smooth hills.

SciPy

scipy.optimize.minimize(fun, x0, method='BFGS')

Searches along directions and works well even if slope is not available.

SciPy

scipy.optimize.minimize(fun, x0, method='Powell')

Sample Program

This code finds the minimum of a simple bowl-shaped function using three methods. It starts from the point (0,0) and prints where each method finds the minimum and the function value there.

SciPy

import numpy as np
from scipy.optimize import minimize

def fun(x):
    return (x[0]-3)**2 + (x[1]+1)**2

x0 = np.array([0, 0])

methods = ['Nelder-Mead', 'BFGS', 'Powell']

for method in methods:
    res = minimize(fun, x0, method=method)
    print(f"Method: {method}")
    print(f"  Minimum at: {res.x}")
    print(f"  Function value: {res.fun}\n")

Important Notes

Nelder-Mead is good when you cannot calculate slopes.

BFGS is faster if you can calculate or approximate slopes.

Powell works well for tricky shapes and does not need slopes.

Summary

Choose Nelder-Mead if you have no slope info.

Choose BFGS if you have slope info and want speed.

Choose Powell for a robust method without slopes.