Nonlinear constraint optimization in SciPy - Step-by-Step Execution

Concept Flow - Nonlinear constraint optimization

Define objective function

↓

Define nonlinear constraints

↓

Choose initial guess

↓

Call optimizer with objective and constraints

↓

Optimizer iterates to find minimum

↓

Check convergence

↓

Return solution

The process starts by defining the function to minimize and constraints, then an optimizer iterates to find the best solution that meets constraints.

Execution Sample

SciPy

from scipy.optimize import minimize

def objective(x):
    return (x[0]-1)**2 + (x[1]-2.5)**2

cons = ({'type': 'ineq', 'fun': lambda x: x[0] - 2*x[1] + 2})

x0 = [2, 0]

res = minimize(objective, x0, constraints=cons)
print(res.x)

This code minimizes a function with one nonlinear inequality constraint starting from an initial guess.

Execution Table

Step	Current x	Objective Value	Constraint Value	Action	Notes
1	[2, 0]	1^2 + (0-2.5)^2 = 6.25	2 - 0 + 2 = 4 >= 0	Evaluate objective and constraint	Initial guess satisfies constraint
2	[1.5, 0.5]	(1.5-1)^2 + (0.5-2.5)^2 = 4.25	1.5 - 1 + 2 = 2.5 >= 0	Move towards minimum	Constraint still satisfied
3	[1.25, 0.75]	(1.25-1)^2 + (0.75-2.5)^2 = 3.06	1.25 - 1.5 + 2 = 1.75 >= 0	Continue optimization	Constraint satisfied
4	[1.0, 1.0]	(1-1)^2 + (1-2.5)^2 = 2.25	1 - 2 + 2 = 1 >= 0	Closer to minimum	Constraint satisfied
5	[0.75, 1.25]	(0.75-1)^2 + (1.25-2.5)^2 = 1.56	0.75 - 2.5 + 2 = 0.25 >= 0	Approaching boundary	Constraint still satisfied
6	[0.7, 1.3]	(0.7-1)^2 + (1.3-2.5)^2 = 1.44	0.7 - 2.6 + 2 = 0.1 >= 0	Near constraint boundary	Constraint satisfied
7	[0.68, 1.34]	(0.68-1)^2 + (1.34-2.5)^2 = 1.42	0.68 - 2.68 + 2 = 0 >= 0	At constraint boundary	Constraint active
8	[0.68, 1.34]	1.42	0	Converged	Solution found at constraint boundary

💡 Optimizer converged when constraint value reached zero and objective minimized

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	After 5	After 6	After 7	Final
x[0]	2	1.5	1.25	1.0	0.75	0.7	0.68	0.68	0.68
x[1]	0	0.5	0.75	1.0	1.25	1.3	1.34	1.34	1.34
Objective	6.25	4.25	3.06	2.25	1.56	1.44	1.42	1.42	1.42
Constraint	4	2.5	1.75	1	0.25	0.1	0	0	0

Key Moments - 2 Insights

Why does the optimizer stop when the constraint value is exactly zero?

Because the constraint is an inequality (>= 0), zero means the boundary is reached, so the solution is feasible and the optimizer stops as it found the minimum on the boundary (see step 7 and 8 in execution_table).

Why does the objective value sometimes increase if the optimizer tries to minimize it?

The optimizer balances minimizing the objective while satisfying constraints. Sometimes it moves slightly away from the minimum to keep constraints valid, as seen in steps where constraint values approach zero but objective decreases more slowly.

Visual Quiz - 3 Questions

Look at the execution_table at step 5, what is the constraint value?

A 1.25

B 0.25

C 0

D 2.5

💡 Hint

Check the 'Constraint Value' column in row for step 5 in execution_table.

At which step does the optimizer reach the constraint boundary (constraint value = 0)?

A Step 7

B Step 4

C Step 3

D Step 2

💡 Hint

Look for the first step where 'Constraint Value' is zero in execution_table.

If the initial guess x0 was [0,0], how would the constraint value at step 1 change?

A 0

B 2

C -2

D 4

💡 Hint

Calculate constraint: x[0] - 2*x[1] + 2 with x=[0,0] and check execution_table logic.

Concept Snapshot

Nonlinear constraint optimization:
- Minimize f(x) subject to constraints g(x) >= 0
- Define objective and constraints as functions
- Use scipy.optimize.minimize with constraints parameter
- Optimizer iterates adjusting x to reduce f(x) while satisfying constraints
- Stops when convergence and constraints met

Full Transcript

Nonlinear constraint optimization finds the minimum of a function while respecting rules called constraints. We start by defining the function to minimize and the constraints it must satisfy. Then we pick a starting point. The optimizer tries different values, checking the function and constraints each time. It moves closer to the minimum but never breaks the constraints. When it finds the best value that meets all constraints, it stops. The example shows steps where the optimizer updates values, checks the objective and constraint, and finally stops when the constraint boundary is reached and the objective is minimized.

Practice

1. What is the main purpose of using nonlinear constraint optimization in scipy.optimize.minimize?

easy

A. To generate random numbers

B. To sort data in ascending order

C. To calculate the mean of a dataset

D. To find the best solution while respecting complex rules or limits

Solution

1. Step 1: Understand the goal of optimization

   Optimization aims to find the best value of a function, often minimum or maximum.

2. Step 2: Recognize the role of constraints

   Nonlinear constraint optimization includes rules that the solution must follow, making it more complex.

3. Final Answer:

   To find the best solution while respecting complex rules or limits -> Option D

4. Quick Check:

   Optimization with constraints = best solution with rules [OK]

Hint: Optimization with constraints means best solution obeying rules [OK]

Common Mistakes:

• Confusing optimization with sorting
• Thinking it calculates statistics like mean
• Assuming it generates random data

2. Which of the following is the correct way to specify nonlinear constraints in scipy.optimize.minimize?

easy

A. constraints = {'ineq', lambda x: x[0] - 1}

B. constraints = ['type' = 'ineq', 'fun' = lambda x: x[0] - 1]

C. constraints = {'type': 'ineq', 'fun': lambda x: x[0] - 1}

D. constraints = ('ineq', lambda x: x[0] - 1)

Solution

1. Step 1: Recall the constraints format

   Constraints must be a dictionary with keys 'type' and 'fun'.

2. Step 2: Check each option's syntax

   constraints = {'type': 'ineq', 'fun': lambda x: x[0] - 1} uses a dictionary with correct keys and lambda function syntax.

3. Final Answer:

   constraints = {'type': 'ineq', 'fun': lambda x: x[0] - 1} -> Option C

4. Quick Check:

   Constraints as dict with 'type' and 'fun' keys = constraints = {'type': 'ineq', 'fun': lambda x: x[0] - 1} [OK]

Hint: Constraints need dict with 'type' and 'fun' keys [OK]

Common Mistakes:

• Using list or tuple instead of dict
• Wrong assignment syntax inside list
• Missing keys or using set instead of dict

3. What is the output of this code snippet?

from scipy.optimize import minimize

obj_fun = lambda x: (x[0]-2)**2 + (x[1]-3)**2
constraint = {'type': 'ineq', 'fun': lambda x: x[0] + x[1] - 4}
result = minimize(obj_fun, [0, 0], constraints=constraint, method='SLSQP')
print(round(result.fun, 2))

medium

A. 0.00

B. 1.00

C. 2.00

D. 4.00

Solution

1. Step 1: Understand the objective function

   The function measures distance squared from point (2,3).

2. Step 2: Apply the constraint and minimize

   The constraint requires x[0] + x[1] >= 4. The closest point to (2,3) on this line is (1,3), giving value (1-2)^2+(3-3)^2=1.

3. Final Answer:

   1.00 -> Option B

4. Quick Check:

   Minimum distance squared with constraint = 1.00 [OK]

Hint: Check closest point on constraint line to target point [OK]

Common Mistakes:

• Ignoring the constraint
• Rounding incorrectly
• Confusing objective function value with variables

4. Identify the error in this code for nonlinear constraint optimization:

from scipy.optimize import minimize

def obj(x):
    return x[0]**2 + x[1]**2

constraint = {'type': 'eq', 'fun': lambda x: x[0] + x[1] - 1}

result = minimize(obj, [0, 0], constraints=constraint, method='SLSQP')
print(result.x)

medium

A. Initial guess violates the equality constraint

B. Constraint type should be 'ineq' instead of 'eq'

C. Objective function must be linear

D. Method 'SLSQP' does not support constraints

Solution

1. Step 1: Check initial guess against constraint

   Initial guess [0,0] does not satisfy x[0] + x[1] = 1.

2. Step 2: Understand impact on solver

   Starting point violating equality constraints can cause solver to fail or converge slowly.

3. Final Answer:

   Initial guess violates the equality constraint -> Option A

4. Quick Check:

   Initial guess must satisfy equality constraints [OK]

Hint: Start with guess satisfying equality constraints [OK]

Common Mistakes:

• Using wrong constraint type
• Assuming objective must be linear
• Thinking SLSQP can't handle constraints

5. You want to minimize f(x) = (x[0]-1)^2 + (x[1]-2)^2 subject to nonlinear constraints x[0]^2 + x[1]^2 <= 2 and x[0] - x[1] >= 0. Which is the correct way to define these constraints for scipy.optimize.minimize with method 'SLSQP'?

hard

A. [{'type': 'ineq', 'fun': lambda x: 2 - (x[0]**2 + x[1]**2)}, {'type': 'ineq', 'fun': lambda x: x[0] - x[1]}]

B. [{'type': 'eq', 'fun': lambda x: 2 - (x[0]**2 + x[1]**2)}, {'type': 'eq', 'fun': lambda x: x[0] - x[1]}]

C. [{'type': 'ineq', 'fun': lambda x: (x[0]**2 + x[1]**2) - 2}, {'type': 'ineq', 'fun': lambda x: x[1] - x[0]}]

D. [{'type': 'ineq', 'fun': lambda x: (x[0]**2 + x[1]**2) - 2}, {'type': 'ineq', 'fun': lambda x: x[0] - x[1]}]

Solution

1. Step 1: Translate constraints to 'ineq' form

   For 'ineq', function must be >= 0. So x0^2+x1^2 <= 2 becomes 2 - (x0^2+x1^2) >= 0.

2. Step 2: Check second constraint

   x0 - x1 >= 0 is already in correct form.

3. Final Answer:

   [{'type': 'ineq', 'fun': lambda x: 2 - (x[0]**2 + x[1]**2)}, {'type': 'ineq', 'fun': lambda x: x[0] - x[1]}] -> Option A

4. Quick Check:

   Constraints must be 'ineq' with function >= 0 [OK]

Hint: Rewrite constraints so function >= 0 for 'ineq' type [OK]

Common Mistakes:

• Using 'eq' instead of 'ineq' for inequalities
• Reversing inequality signs
• Not rewriting constraints to >= 0 form