Fitting custom models in SciPy - Time & Space Complexity

Time Complexity: Fitting custom models

O(n)

Understanding Time Complexity

When fitting custom models using scipy, it is important to understand how the time needed grows as the data size increases.

We want to know how the number of calculations changes when we fit models to more data points.

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import numpy as np
from scipy.optimize import curve_fit

def model_func(x, a, b):
    return a * np.exp(b * x)

xdata = np.linspace(0, 4, 50)
ydata = model_func(xdata, 2.5, 1.3) + 0.2 * np.random.normal(size=xdata.size)

popt, pcov = curve_fit(model_func, xdata, ydata)

This code fits a custom exponential model to data points using scipy's curve_fit function.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Evaluating the model function many times during optimization.
• How many times: The optimizer calls the model function repeatedly, often hundreds of times, depending on convergence.

How Execution Grows With Input

As the number of data points increases, the model function must be evaluated on each point every time the optimizer tries new parameters.

Input Size (n)	Approx. Operations
10	Few hundred evaluations x 10 points = ~thousands
100	Few hundred evaluations x 100 points = ~tens of thousands
1000	Few hundred evaluations x 1000 points = ~hundreds of thousands

Pattern observation: The total work grows roughly linearly with the number of data points.

Final Time Complexity

Time Complexity: O(n)

This means the time to fit the model grows roughly in direct proportion to the number of data points.

Common Mistake

[X] Wrong: "The fitting time depends only on the number of parameters, not the data size."

[OK] Correct: Each fitting step evaluates the model on all data points, so more data means more work even if parameters stay the same.

Interview Connect

Understanding how fitting time grows helps you explain performance in real projects and shows you can reason about algorithm costs beyond just writing code.

Self-Check

"What if the model function was very complex and took longer to compute per point? How would that affect the overall time complexity?"

Practice

1. What is the main purpose of using scipy.optimize.curve_fit in fitting custom models?

easy

A. To find the best parameters that make the model fit the data

B. To plot the data points automatically

C. To generate random data for testing

D. To calculate the mean of the dataset

Solution

1. Step 1: Understand the role of curve_fit

   curve_fit is used to adjust parameters of a model function so that it best fits the given data points.

2. Step 2: Identify the correct purpose

   It does not plot data, generate random data, or calculate means. Its main job is parameter estimation for fitting.

3. Final Answer:

   To find the best parameters that make the model fit the data -> Option A

4. Quick Check:

   curve_fit finds best parameters [OK]

Hint: Remember: curve_fit adjusts parameters to fit data [OK]

Common Mistakes:

• Thinking curve_fit plots data automatically
• Confusing curve_fit with data generation functions
• Assuming curve_fit calculates statistics like mean

2. Which of the following is the correct way to define a custom model function for curve_fit that fits a line y = m*x + c?

easy

A. def model(m, c, x): return m + c * x

B. def model(x, m, c): return m * x + c

C. def model(x): return m * x + c

D. def model(x, m, c): return m + c / x

Solution

1. Step 1: Check parameter order for curve_fit

   The model function must have the independent variable as the first argument, followed by parameters to fit.

2. Step 2: Verify function matches y = m*x + c

   def model(x, m, c): return m * x + c correctly defines model(x, m, c) returning m * x + c. Others have wrong order or formula.

3. Final Answer:

   def model(x, m, c): return m * x + c -> Option B

4. Quick Check:

   Model args: x first, then parameters [OK]

Hint: Model function: x first, then parameters [OK]

Common Mistakes:

• Swapping parameter and variable order
• Missing parameters in function definition
• Using wrong formula inside the function

3. Given the code below, what will be the output of print(popt)?

import numpy as np
from scipy.optimize import curve_fit

def model(x, a, b):
    return a * np.exp(b * x)

xdata = np.array([0, 1, 2, 3])
ydata = np.array([1, 2.7, 7.4, 20.1])

popt, _ = curve_fit(model, xdata, ydata)
print(np.round(popt, 2))

medium

A. [1.0 0.99]

B. [1.0 1.0]

C. [1.0 1.0] but with a runtime error

D. [1.01 1.0]

Solution

1. Step 1: Understand the model and data

   The model is a * exp(b * x). Given ydata roughly follows exponential growth, parameters a and b will be close to 1.

2. Step 2: Check output of curve_fit

   Running the code fits parameters close to a=1.0 and b=0.99 (data approximates e^x but slightly less). Rounded to two decimals, popt is approximately [1.0 0.99].

3. Final Answer:

   [1.0 0.99] -> Option A

4. Quick Check:

   Exponential fit params ~ [1.0, 0.99] [OK]

Hint: Run curve_fit and round parameters to check values [OK]

Common Mistakes:

• Misestimating parameters as [1.0 1.0] due to data approximation
• Confusing parameter order
• Expecting runtime errors without cause

4. What is wrong with the following code snippet for fitting a quadratic model using curve_fit?

import numpy as np
from scipy.optimize import curve_fit

def quad(x, a, b, c):
    return a * x**2 + b * x + c

xdata = np.array([1, 2, 3, 4])
ydata = np.array([3, 7, 13, 21])

popt, pcov = curve_fit(quad, ydata, xdata)
print(popt)

medium

A. Missing initial guess for parameters

B. The model function has wrong formula for quadratic

C. The independent and dependent variables are swapped in curve_fit call

D. The print statement is incorrect

Solution

1. Step 1: Check curve_fit arguments

   curve_fit expects the model, xdata (independent), then ydata (dependent). Here, ydata and xdata are swapped.

2. Step 2: Identify the error impact

   Swapping causes wrong fitting or runtime errors because the model expects x values first.

3. Final Answer:

   The independent and dependent variables are swapped in curve_fit call -> Option C

4. Quick Check:

   curve_fit(xdata, ydata) order matters [OK]

Hint: Remember: curve_fit(model, xdata, ydata) [OK]

Common Mistakes:

• Swapping xdata and ydata in curve_fit
• Assuming model formula is incorrect
• Thinking initial guess is always required

5. You want to fit a custom model y = a * sin(b * x) + c to noisy data. Which approach correctly fits the model and plots the result?

hard

A. Plot data first, then call curve_fit without storing parameters

B. Use curve_fit without defining a model function, just pass np.sin

C. Fit the model by manually guessing parameters without curve_fit

D. Define model with def model(x, a, b, c): return a * np.sin(b * x) + c, use curve_fit with data, then plot original and fitted curves

Solution

1. Step 1: Define the correct model function

   Model must be defined as model(x, a, b, c) returning a * np.sin(b * x) + c.

2. Step 2: Use curve_fit and plot results

   Call curve_fit(model, xdata, ydata) to get parameters, then plot original data and fitted curve for comparison.

3. Final Answer:

   Define model with def model(x, a, b, c): return a * np.sin(b * x) + c, use curve_fit with data, then plot original and fitted curves -> Option D

4. Quick Check:

   Model function + curve_fit + plot = correct approach [OK]

Hint: Always define model function before curve_fit and plot results [OK]

Common Mistakes:

• Passing np.sin directly without parameters
• Skipping model function definition
• Not storing or using fitted parameters for plotting