Bounds and constraints in SciPy - Time & Space Complexity

Time Complexity: Bounds and constraints

O(n^2)

Understanding Time Complexity

When using bounds and constraints in optimization with scipy, it is important to know how the time to find a solution changes as the problem size grows.

We want to understand how adding bounds and constraints affects the work the solver must do.

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

from scipy.optimize import minimize

# Objective function
fun = lambda x: (x[0] - 1)**2 + (x[1] - 2.5)**2

# Bounds for variables
bounds = [(0, 2), (0, 3)]

# Constraint: x0 + x1 must be at least 2
cons = ({'type': 'ineq', 'fun': lambda x: x[0] + x[1] - 2})

# Run optimization
result = minimize(fun, x0=[0, 0], bounds=bounds, constraints=[cons])

This code finds the minimum of a function with two variables, using bounds and one inequality constraint.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: The solver repeatedly evaluates the objective and constraint functions while searching for the minimum.
• How many times: The number of evaluations depends on the solver's iterations, which grow with problem size and complexity.

How Execution Grows With Input

As the number of variables and constraints increases, the solver must do more work to check bounds and satisfy constraints.

Input Size (n variables)	Approx. Operations
10	Hundreds of function and constraint checks
100	Thousands to tens of thousands of checks
1000	Hundreds of thousands or more checks

Pattern observation: The work grows faster than the number of variables because constraints add extra checks each iteration.

Final Time Complexity

Time Complexity: O(n^2)

This means the time to solve grows roughly with the square of the number of variables when bounds and constraints are involved.

Common Mistake

[X] Wrong: "Adding bounds or constraints does not affect the solver's time much."

[OK] Correct: Bounds and constraints require extra checks and calculations each iteration, increasing the total work significantly.

Interview Connect

Understanding how constraints affect solver time helps you explain trade-offs in optimization problems clearly and confidently.

Self-Check

"What if we remove all constraints but keep bounds? How would the time complexity change?"