Nonlinear constraint optimization in SciPy

Introduction

We use nonlinear constraint optimization to find the best solution when the problem has limits that are not straight lines. It helps us solve real problems with complex rules.

Planning the best route for delivery trucks with speed and road limits

Finding the best mix of ingredients in a recipe with taste and nutrition rules

Designing a product that must meet safety and size limits

Allocating budget to projects with minimum and maximum spending rules

Syntax

SciPy

from scipy.optimize import minimize

result = minimize(
    fun,               # The function to minimize
    x0,                # Starting guess for variables
    constraints=[      # List of constraints
        {'type': 'eq', 'fun': constraint_eq},   # Equality constraint
        {'type': 'ineq', 'fun': constraint_ineq} # Inequality constraint
    ],
    method='SLSQP'     # Optimization method that handles constraints
)

fun is the function you want to minimize.

Constraints can be equality (must be zero) or inequality (must be ≥ 0).

Examples

Minimize the sum of squares with the rule that x + y = 1.

SciPy

def fun(x):
    return x[0]**2 + x[1]**2

def constraint_eq(x):
    return x[0] + x[1] - 1

result = minimize(fun, [0, 0], constraints=[{'type': 'eq', 'fun': constraint_eq}], method='SLSQP')

Minimize distance to point (1, 2.5) with the rule x - 2y + 2 ≥ 0.

SciPy

def fun(x):
    return (x[0]-1)**2 + (x[1]-2.5)**2

def constraint_ineq(x):
    return x[0] - 2*x[1] + 2

result = minimize(fun, [2, 0], constraints=[{'type': 'ineq', 'fun': constraint_ineq}], method='SLSQP')

Sample Program

This code finds the point (x, y) closest to the origin (0,0) that satisfies two rules: x + y = 1 and x - y ≥ 1.

SciPy

from scipy.optimize import minimize

# Objective function: minimize x^2 + y^2
def objective(x):
    return x[0]**2 + x[1]**2

# Equality constraint: x + y = 1
def eq_constraint(x):
    return x[0] + x[1] - 1

# Inequality constraint: x - y >= 1  (rewritten as x - y - 1 >= 0)
def ineq_constraint(x):
    return x[0] - x[1] - 1

constraints = [
    {'type': 'eq', 'fun': eq_constraint},
    {'type': 'ineq', 'fun': ineq_constraint}
]

# Starting guess
x0 = [0, 0]

result = minimize(objective, x0, constraints=constraints, method='SLSQP')

print('Optimal solution:', result.x)
print('Objective value:', result.fun)

Important Notes

The method 'SLSQP' is good for problems with nonlinear constraints.

Always provide a reasonable starting guess to help the solver find the best answer.

Constraints functions must return zero for equality and non-negative for inequality.

Summary

Nonlinear constraint optimization finds the best solution with complex rules.

Use scipy.optimize.minimize with constraints and method 'SLSQP'.

Define your objective and constraint functions clearly for the solver.

Practice

1. What is the main purpose of using nonlinear constraint optimization in scipy.optimize.minimize?

easy

A. To generate random numbers

B. To sort data in ascending order

C. To calculate the mean of a dataset

D. To find the best solution while respecting complex rules or limits

Solution

1. Step 1: Understand the goal of optimization

   Optimization aims to find the best value of a function, often minimum or maximum.

2. Step 2: Recognize the role of constraints

   Nonlinear constraint optimization includes rules that the solution must follow, making it more complex.

3. Final Answer:

   To find the best solution while respecting complex rules or limits -> Option D

4. Quick Check:

   Optimization with constraints = best solution with rules [OK]

Hint: Optimization with constraints means best solution obeying rules [OK]

Common Mistakes:

• Confusing optimization with sorting
• Thinking it calculates statistics like mean
• Assuming it generates random data

2. Which of the following is the correct way to specify nonlinear constraints in scipy.optimize.minimize?

easy

A. constraints = {'ineq', lambda x: x[0] - 1}

B. constraints = ['type' = 'ineq', 'fun' = lambda x: x[0] - 1]

C. constraints = {'type': 'ineq', 'fun': lambda x: x[0] - 1}

D. constraints = ('ineq', lambda x: x[0] - 1)

Solution

1. Step 1: Recall the constraints format

   Constraints must be a dictionary with keys 'type' and 'fun'.

2. Step 2: Check each option's syntax

   constraints = {'type': 'ineq', 'fun': lambda x: x[0] - 1} uses a dictionary with correct keys and lambda function syntax.

3. Final Answer:

   constraints = {'type': 'ineq', 'fun': lambda x: x[0] - 1} -> Option C

4. Quick Check:

   Constraints as dict with 'type' and 'fun' keys = constraints = {'type': 'ineq', 'fun': lambda x: x[0] - 1} [OK]

Hint: Constraints need dict with 'type' and 'fun' keys [OK]

Common Mistakes:

• Using list or tuple instead of dict
• Wrong assignment syntax inside list
• Missing keys or using set instead of dict

3. What is the output of this code snippet?

from scipy.optimize import minimize

obj_fun = lambda x: (x[0]-2)**2 + (x[1]-3)**2
constraint = {'type': 'ineq', 'fun': lambda x: x[0] + x[1] - 4}
result = minimize(obj_fun, [0, 0], constraints=constraint, method='SLSQP')
print(round(result.fun, 2))

medium

A. 0.00

B. 1.00

C. 2.00

D. 4.00

Solution

1. Step 1: Understand the objective function

   The function measures distance squared from point (2,3).

2. Step 2: Apply the constraint and minimize

   The constraint requires x[0] + x[1] >= 4. The closest point to (2,3) on this line is (1,3), giving value (1-2)^2+(3-3)^2=1.

3. Final Answer:

   1.00 -> Option B

4. Quick Check:

   Minimum distance squared with constraint = 1.00 [OK]

Hint: Check closest point on constraint line to target point [OK]

Common Mistakes:

• Ignoring the constraint
• Rounding incorrectly
• Confusing objective function value with variables

4. Identify the error in this code for nonlinear constraint optimization:

from scipy.optimize import minimize

def obj(x):
    return x[0]**2 + x[1]**2

constraint = {'type': 'eq', 'fun': lambda x: x[0] + x[1] - 1}

result = minimize(obj, [0, 0], constraints=constraint, method='SLSQP')
print(result.x)

medium

A. Initial guess violates the equality constraint

B. Constraint type should be 'ineq' instead of 'eq'

C. Objective function must be linear

D. Method 'SLSQP' does not support constraints

Solution

1. Step 1: Check initial guess against constraint

   Initial guess [0,0] does not satisfy x[0] + x[1] = 1.

2. Step 2: Understand impact on solver

   Starting point violating equality constraints can cause solver to fail or converge slowly.

3. Final Answer:

   Initial guess violates the equality constraint -> Option A

4. Quick Check:

   Initial guess must satisfy equality constraints [OK]

Hint: Start with guess satisfying equality constraints [OK]

Common Mistakes:

• Using wrong constraint type
• Assuming objective must be linear
• Thinking SLSQP can't handle constraints

5. You want to minimize f(x) = (x[0]-1)^2 + (x[1]-2)^2 subject to nonlinear constraints x[0]^2 + x[1]^2 <= 2 and x[0] - x[1] >= 0. Which is the correct way to define these constraints for scipy.optimize.minimize with method 'SLSQP'?

hard

A. [{'type': 'ineq', 'fun': lambda x: 2 - (x[0]**2 + x[1]**2)}, {'type': 'ineq', 'fun': lambda x: x[0] - x[1]}]

B. [{'type': 'eq', 'fun': lambda x: 2 - (x[0]**2 + x[1]**2)}, {'type': 'eq', 'fun': lambda x: x[0] - x[1]}]

C. [{'type': 'ineq', 'fun': lambda x: (x[0]**2 + x[1]**2) - 2}, {'type': 'ineq', 'fun': lambda x: x[1] - x[0]}]

D. [{'type': 'ineq', 'fun': lambda x: (x[0]**2 + x[1]**2) - 2}, {'type': 'ineq', 'fun': lambda x: x[0] - x[1]}]

Solution

1. Step 1: Translate constraints to 'ineq' form

   For 'ineq', function must be >= 0. So x0^2+x1^2 <= 2 becomes 2 - (x0^2+x1^2) >= 0.

2. Step 2: Check second constraint

   x0 - x1 >= 0 is already in correct form.

3. Final Answer:

   [{'type': 'ineq', 'fun': lambda x: 2 - (x[0]**2 + x[1]**2)}, {'type': 'ineq', 'fun': lambda x: x[0] - x[1]}] -> Option A

4. Quick Check:

   Constraints must be 'ineq' with function >= 0 [OK]

Hint: Rewrite constraints so function >= 0 for 'ineq' type [OK]

Common Mistakes:

• Using 'eq' instead of 'ineq' for inequalities
• Reversing inequality signs
• Not rewriting constraints to >= 0 form