Least squares (least_squares) in SciPy

Introduction

Least squares helps find the best fit line or curve to data by minimizing errors. It makes predictions closer to actual points.

You want to find a line that best fits scattered data points.

You need to estimate parameters in a model from noisy measurements.

You want to solve equations that don't have exact solutions.

You want to reduce the difference between predicted and actual values.

You want to analyze trends in data with some randomness.

Syntax

SciPy

scipy.optimize.least_squares(fun, x0, args=(), method='trf', jac='2-point', bounds=(-np.inf, np.inf), ...)

fun is the function that calculates residuals (differences).

x0 is the initial guess for the parameters to find.

Examples

Finds x where x² - 4 is close to zero, starting from 1.

SciPy

from scipy.optimize import least_squares

def fun(x):
    return x**2 - 4

result = least_squares(fun, x0=[1])

Fits a line y = m*x + b to points (1,2), (2,4), (3,6).

SciPy

def residuals(params, x, y):
    return params[0] * x + params[1] - y

x_data = [1, 2, 3]
y_data = [2, 4, 6]

result = least_squares(residuals, x0=[0, 0], args=(x_data, y_data))

Sample Program

This code fits a straight line to points that roughly follow y = 2x + 1. It finds the best slope (m) and intercept (b) that minimize the difference between the line and points.

SciPy

from scipy.optimize import least_squares
import numpy as np

# Define residuals function for line y = m*x + b
# params = [m, b]
def residuals(params, x, y):
    m, b = params
    return m * x + b - y

# Sample data points
x_data = np.array([0, 1, 2, 3, 4])
y_data = np.array([1, 3, 5, 7, 9])  # roughly y = 2*x + 1

# Initial guess for m and b
initial_guess = [0, 0]

# Run least squares to fit line
result = least_squares(residuals, initial_guess, args=(x_data, y_data))

# Extract fitted parameters
m_fit, b_fit = result.x

print(f"Fitted line: y = {m_fit:.2f}*x + {b_fit:.2f}")

Important Notes

Least squares finds parameters that make residuals (differences) as small as possible.

Initial guess affects how fast and well the solution is found.

It works well when the model is close to linear or smooth.

Summary

Least squares finds the best fit by minimizing errors between model and data.

Use scipy.optimize.least_squares with a residual function and initial guess.

It helps estimate parameters for models from noisy or imperfect data.