Least squares (least_squares) in SciPy - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of scipy.optimize.least_squares with linear residuals

What is the output of the following code snippet using scipy.optimize.least_squares to fit a line y = mx + b to data points?

SciPy

import numpy as np
from scipy.optimize import least_squares

def residuals(params, x, y):
    m, b = params
    return m * x + b - y

x_data = np.array([0, 1, 2, 3])
y_data = np.array([1, 3, 5, 7])

result = least_squares(residuals, x0=[0, 0], args=(x_data, y_data))
print(result.x)

A [3.0, 0.0]

B [1.0, 2.0]

C [0.5, 1.0]

D [2.0, 1.0]

💡 Hint

Think about the line that fits points (0,1), (1,3), (2,5), (3,7) exactly.

The data points lie perfectly on the line y = 2x + 1. The least squares method finds parameters m=2 and b=1.

❓ data_output

intermediate

Number of iterations in least_squares optimization

After running the following code, what is the value of result.nfev (number of function evaluations)?

SciPy

import numpy as np
from scipy.optimize import least_squares

def residuals(p, x, y):
    return p[0] * x + p[1] - y

x = np.linspace(0, 10, 5)
y = 3 * x + 4 + np.random.normal(0, 0.1, size=x.size)

result = least_squares(residuals, x0=[0, 0], args=(x, y))
print(result.nfev)

A 10

B 4

C 6

D 1

💡 Hint

The solver usually needs multiple evaluations to converge, but not too many for a simple linear fit.

The solver typically evaluates the residual function about 6 times for this simple problem before converging.

🔧 Debug

advanced

Identify the error in least_squares residual function

What error will this code raise when run?

SciPy

import numpy as np
from scipy.optimize import least_squares

def residuals(params, x, y):
    m, b = params
    return m * x + b + y  # Incorrect sign here

x = np.array([1, 2, 3])
y = np.array([2, 4, 6])

result = least_squares(residuals, x0=[0, 0], args=(x, y))

A No error, but wrong fit result

B TypeError due to incompatible operations

C ValueError due to shape mismatch

D RuntimeWarning due to overflow

💡 Hint

Check the residuals formula carefully for correctness.

The code runs without error but the residuals are incorrect because it adds y instead of subtracting, leading to a wrong fit.

🧠 Conceptual

advanced

Effect of jac='3-point' in least_squares

What does setting jac='3-point' do in scipy.optimize.least_squares?

A Uses an analytical Jacobian provided by the user

B Uses a numerical approximation of the Jacobian using three points per variable

C Disables Jacobian calculation for faster runtime

D Uses a random Jacobian matrix for stochastic optimization

💡 Hint

Think about numerical differentiation methods.

The '3-point' option uses a finite difference method with three points to approximate the Jacobian numerically.

🚀 Application

expert

Predict parameter values from least_squares output

Given this output from least_squares: result.x = [1.5, -0.5], and residuals defined as m * x + b - y, what is the predicted y value for x=4?

A 5.5

B 6.5

C 7.5

D 4.5

💡 Hint

Use the formula y = m*x + b with given parameters.

With m=1.5 and b=-0.5, y = 1.5*4 - 0.5 = 6 - 0.5 = 5.5.