Least squares optimization in SciPy

Introduction

Least squares optimization helps find the best fit line or curve to data by minimizing the total error. It makes predictions closer to actual points.

You want to fit a straight line to data points to see a trend.

You need to find parameters of a model that best explain your data.

You want to reduce errors between predicted and actual values in a dataset.

You want to smooth noisy data by fitting a curve.

You want to estimate unknown values from observed data.

Syntax

SciPy

from scipy.optimize import least_squares

result = least_squares(fun, x0, args=(), kwargs=None, method='trf')

fun is the function that calculates residuals (differences) between model and data.

x0 is the initial guess for the parameters to optimize.

Examples

Fit a line y = a*x + b to data by minimizing residuals.

SciPy

def residuals(params):
    a, b = params
    return a * x_data + b - y_data

result = least_squares(residuals, x0=[1, 0])

Fit a quadratic curve to data using extra arguments.

SciPy

def residuals(params, x, y):
    return params[0] * x ** 2 + params[1] * x + params[2] - y

result = least_squares(residuals, x0=[1, 1, 0], args=(x_data, y_data))

Sample Program

This code fits a quadratic curve to the sample data points by minimizing the difference between the curve and the data.

SciPy

import numpy as np
from scipy.optimize import least_squares

# Sample data points
x_data = np.array([0, 1, 2, 3, 4, 5])
y_data = np.array([1, 3, 7, 13, 21, 31])

# Define residuals function for model y = a*x^2 + b*x + c
def residuals(params, x, y):
    a, b, c = params
    return a * x**2 + b * x + c - y

# Initial guess for parameters a, b, c
initial_guess = [1, 1, 1]

# Run least squares optimization
result = least_squares(residuals, initial_guess, args=(x_data, y_data))

# Print optimized parameters
print(f"Optimized parameters: a={result.x[0]:.2f}, b={result.x[1]:.2f}, c={result.x[2]:.2f}")

Important Notes

The function you minimize should return residuals, not the sum of squares.

Good initial guesses help the optimizer find the best solution faster.

Least squares works well when errors are normally distributed and small.

Summary

Least squares optimization finds parameters that minimize the difference between model and data.

Use scipy.optimize.least_squares by defining a residuals function and giving an initial guess.

This method helps fit lines, curves, or complex models to data.