Constrained optimization in SciPy - Step-by-Step Execution

Concept Flow - Constrained optimization

Define objective function

↓

Define constraints

↓

Choose initial guess

↓

Call optimizer

↓

Check constraints satisfied?

No→Adjust and retry

Yes↓

Return optimal solution

The process starts by defining the function to minimize and constraints, then uses an optimizer to find the best solution that meets all constraints.

Execution Sample

SciPy

from scipy.optimize import minimize

# Objective: minimize x^2 + y^2
# Constraint: x + y = 1

res = minimize(lambda v: v[0]**2 + v[1]**2, [0,0], constraints={'type':'eq','fun': lambda v: v[0]+v[1]-1})
print(res.x)

This code finds the point (x,y) closest to origin where x + y = 1.

Execution Table

Step	Current guess (x,y)	Objective value	Constraint value (x+y-1)	Action
1	[0, 0]	0	-1	Start at initial guess, constraint not met
2	[0.5, 0.5]	0.5	0	Constraint met, objective improved
3	[0.6, 0.4]	0.52	0	Constraint met, objective worse, optimizer tries better
4	[0.5, 0.5]	0.5	0	Optimizer returns best found solution
Exit	-	-	-	Optimization converged with constraint satisfied

💡 Optimization stops when constraint is met and objective cannot be improved further.

Variable Tracker

Variable	Start	After 1	After 2	After 3	Final
x	0	0.5	0.6	0.5	0.5
y	0	0.5	0.4	0.5	0.5
Objective	0	0.5	0.52	0.5	0.5
Constraint	-1	0	0	0	0

Key Moments - 2 Insights

Why does the optimizer start with a guess that does not satisfy the constraint?

The optimizer begins with the initial guess provided (0,0) even if it violates constraints. It then adjusts guesses to satisfy constraints as shown in steps 1 to 2 in the execution table.

Why can the objective value increase in some steps during optimization?

Sometimes the optimizer tries points with worse objective values temporarily to find a better overall solution that satisfies constraints, as seen in step 3 where objective increased but constraint was met.

Visual Quiz - 3 Questions

Look at the execution table, what is the constraint value at step 2?

A -1

B 0

C 0.5

D 1

💡 Hint

Check the 'Constraint value' column at step 2 in the execution_table.

At which step does the optimizer first meet the constraint exactly?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Look for the first step where 'Constraint value' is zero in the execution_table.

If the initial guess was [1,1], how would the constraint value at start change?

A It would be 1

B It would be 0

C It would be 2

D It would be -1

💡 Hint

Constraint is x + y - 1, so sum the initial guess values and subtract 1.

Concept Snapshot

Constrained optimization:
- Minimize a function subject to constraints
- Define objective and constraints
- Provide initial guess
- Use scipy.optimize.minimize with constraints
- Optimizer adjusts guess to satisfy constraints
- Returns solution meeting constraints with minimal objective

Full Transcript

Constrained optimization finds the best solution that meets rules. We start with a guess, check if it meets constraints, and improve it step by step. The optimizer tries different points, sometimes worse, to find the best valid answer. The process stops when constraints are met and no better solution is found.