Global optimization (differential_evolution) in SciPy - Step-by-Step Execution

Concept Flow - Global optimization (differential_evolution)

Define objective function

↓

Set bounds for variables

↓

Initialize population randomly

↓

Mutation and recombination

↓

Selection: choose better solutions

↓

Check stopping criteria

↓

Return best

The differential evolution algorithm starts with a random population, then improves solutions by mutation, recombination, and selection until stopping criteria are met.

Execution Sample

SciPy

from scipy.optimize import differential_evolution

def f(x):
    return (x[0]-3)**2 + (x[1]+1)**2

bounds = [(-5, 5), (-5, 5)]
result = differential_evolution(f, bounds)
print(result.x, result.fun)

This code finds the minimum of a simple function using differential evolution within given bounds.

Execution Table

Step	Population Sample	Mutation Vector	Trial Vector	Selection Result	Best Solution So Far
1	[[0.1, -4.5], [2.0, 0.0], [-3.0, 1.0], [4.5, -2.0]]	[2.0, 0.0]	[2.0, 0.0]	Selected (better)	[2.0, 0.0] (fun=2)
2	[[2.0, 0.0], [2.5, -1.0], [-3.0, 1.0], [4.5, -2.0]]	[2.5, -1.0]	[2.5, -1.0]	Selected (better)	[2.5, -1.0] (fun=0.25)
3	[[2.0, 0.0], [-3.0, 1.0], [3.0, -1.0], [4.5, -2.0]]	[3.0, -1.0]	[3.0, -1.0]	Selected (better)	[3.0, -1.0] (fun=0)
4	[[3.0, -1.0], [-3.0, 1.0], [3.0, -1.0], [4.5, -2.0]]	[4.5, -2.0]	[4.5, -2.0]	Not selected (worse)	[3.0, -1.0] (fun=0)
5	[[3.0, -1.0], [-3.0, 1.0], [3.0, -1.0], [4.5, -2.0]]	[2.9, -1.1]	[2.9, -1.1]	Selected (better)	[2.9, -1.1] (fun=0.02)
Exit	-	-	-	-	Converged to best solution [2.9, -1.1] with fun=0.02

💡 Algorithm stops when improvement is minimal or max iterations reached.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	After 5	Final
Population	[[0.1, -4.5], [2.0, 0.0], [-3.0, 1.0], [4.5, -2.0]]	[[2.0, 0.0], [2.5, -1.0], [-3.0, 1.0], [4.5, -2.0]]	[[2.0, 0.0], [-3.0, 1.0], [3.0, -1.0], [4.5, -2.0]]	[[3.0, -1.0], [-3.0, 1.0], [3.0, -1.0], [4.5, -2.0]]	[[3.0, -1.0], [-3.0, 1.0], [3.0, -1.0], [4.5, -2.0]]	[[3.0, -1.0], [2.9, -1.1], [3.0, -1.0], [4.5, -2.0]]	[[3.0, -1.0], [2.9, -1.1], [3.0, -1.0], [4.5, -2.0]]
Best Solution	None	[2.0, 0.0]	[2.5, -1.0]	[3.0, -1.0]	[3.0, -1.0]	[2.9, -1.1]	[2.9, -1.1]
Best Fun Value	None	2	0.25	0	0	0.02	0.02

Key Moments - 3 Insights

Why does the algorithm sometimes keep a worse solution instead of the trial vector?

Because selection only replaces a population member if the trial vector has a better (lower) function value, as shown in execution_table step 4 where the trial vector was worse and not selected.

How does the population change over iterations?

The population updates by replacing individuals with better trial vectors, as seen in variable_tracker where population members change after each iteration.

What stops the differential evolution algorithm?

The algorithm stops when improvements become very small or after a max number of iterations, as noted in the exit_note and the last row of execution_table.

Visual Quiz - 3 Questions

Look at the execution_table at step 3. What is the best solution and its function value?

A [2.0, 0.0] with fun=2

B [-3.0, 1.0] with fun=40

C [3.0, -1.0] with fun=0

D [4.5, -2.0] with fun=unknown

💡 Hint

Check the 'Best Solution So Far' column at step 3 in execution_table.

At which step does the algorithm reject a trial vector because it is worse?

A Step 4

B Step 2

C Step 5

D Step 1

💡 Hint

Look for 'Not selected (worse)' in the Selection Result column in execution_table.

If the bounds were changed to narrower ranges, how would the population likely change?

A Population would become larger

B Population values would be limited within new bounds

C Population would not change

D Algorithm would stop immediately

💡 Hint

Variable_tracker shows population values; bounds restrict possible values.

Concept Snapshot

differential_evolution(func, bounds)
- Starts with random population within bounds
- Uses mutation and recombination to create trial vectors
- Selects better solutions to form next generation
- Stops when convergence or max iterations reached
- Returns best solution found and its function value

Full Transcript

Differential evolution is a global optimization method. It starts by defining the function to minimize and setting variable bounds. Then it creates a random population of candidate solutions. Each iteration, it mutates and recombines population members to create trial vectors. If a trial vector improves the function value, it replaces the original member. This process repeats until the solution converges or a maximum number of iterations is reached. The best solution and its function value are returned.

Practice

1. What is the main purpose of the differential_evolution function in scipy.optimize?

easy

A. To perform linear regression on data

B. To calculate the derivative of a function

C. To find the global minimum of a function within given bounds

D. To generate random numbers for simulations

Solution

1. Step 1: Understand the purpose of differential evolution

   Differential evolution is a global optimization algorithm designed to find the minimum value of a function over a specified range.

2. Step 2: Compare options with the purpose

   Only To find the global minimum of a function within given bounds describes finding the global minimum within bounds, which matches the function's purpose.

3. Final Answer:

   To find the global minimum of a function within given bounds -> Option C

4. Quick Check:

   Global optimization = find minimum [OK]

Hint: Remember: differential_evolution finds global minimum [OK]

Common Mistakes:

• Confusing optimization with differentiation
• Thinking it performs regression
• Assuming it generates random numbers only

2. Which of the following is the correct way to import the differential_evolution function from scipy.optimize?

easy

A. import differential_evolution from scipy.optimize

B. from scipy.optimize import differential_evolution

C. from scipy import differential_evolution.optimize

D. import scipy.optimize.differential_evolution

Solution

1. Step 1: Recall Python import syntax

   The correct syntax to import a function from a module is: from module import function.

2. Step 2: Match syntax with options

   from scipy.optimize import differential_evolution matches the correct syntax exactly. Other options have incorrect order or syntax.

3. Final Answer:

   from scipy.optimize import differential_evolution -> Option B

4. Quick Check:

   Correct import syntax = from scipy.optimize import differential_evolution [OK]

Hint: Use 'from module import function' for importing functions [OK]

Common Mistakes:

• Using 'import function from module' which is invalid
• Trying to import submodules incorrectly
• Using dot notation in import statements wrongly

3. What will be the output of the following code snippet?

from scipy.optimize import differential_evolution

def func(x):
    return (x[0] - 2)**2 + (x[1] + 3)**2

result = differential_evolution(func, bounds=[(-5, 5), (-5, 5)])
print(round(result.fun, 2))

medium

A. 0.00

B. 5.00

C. 13.00

D. Error due to wrong bounds

Solution

1. Step 1: Understand the function and bounds

   The function calculates the sum of squares of (x0-2) and (x1+3). The minimum is at x0=2, x1=-3, which is inside the bounds (-5,5).

2. Step 2: Evaluate the minimum function value

   At x0=2 and x1=-3, func returns (2-2)^2 + (-3+3)^2 = 0 + 0 = 0. So the minimum function value is 0.

3. Final Answer:

   0.00 -> Option A

4. Quick Check:

   Minimum value at (2,-3) = 0 [OK]

Hint: Minimum of squared terms is zero at exact target points [OK]

Common Mistakes:

• Misunderstanding bounds causing error
• Calculating function value at wrong points
• Ignoring rounding in output

4. Identify the error in the following code using differential_evolution and how to fix it:

from scipy.optimize import differential_evolution

def f(x):
    return x**2

bounds = [(-3, 3)]
result = differential_evolution(f, bounds)
print(result.x)

medium

A. No error; code runs correctly

B. Bounds should be a tuple, not a list; fix by changing bounds to a tuple

C. Missing import of numpy; fix by adding import numpy as np

D. Function f expects a scalar but receives an array; fix by defining f(x) to accept array input

Solution

1. Step 1: Analyze function input requirements

   The differential_evolution function passes an array to f, but f expects a scalar and tries to square it directly, which causes an error.

2. Step 2: Fix function to accept array input

   Modify f to accept an array input and return a scalar, for example: def f(x): return x[0]**2.

3. Final Answer:

   Function f expects a scalar but receives an array; fix by defining f(x) to accept array input -> Option D

4. Quick Check:

   Function input shape mismatch = Function f expects a scalar but receives an array; fix by defining f(x) to accept array input [OK]

Hint: Functions must accept array input for differential_evolution [OK]

Common Mistakes:

• Assuming function gets scalar input
• Incorrect bounds format
• Ignoring error messages about input types

5. You want to minimize the function f(x) = sin(x[0]) + cos(x[1]) over the bounds x[0] in [0, 6] and x[1] in [-3, 3]. Which of the following code snippets correctly uses differential_evolution to find the minimum and print the best input values?

hard

A. from scipy.optimize import differential_evolution
import numpy as np

def f(x):
return np.sin(x[0]) + np.cos(x[1])

bounds = [(0, 6), (-3, 3)]
result = differential_evolution(f, bounds)
print(result.x)

B. from scipy.optimize import differential_evolution
import numpy as np

def f(x):
return np.sin(x) + np.cos(x)

bounds = [(0, 6), (-3, 3)]
result = differential_evolution(f, bounds)
print(result.x)

C. from scipy.optimize import differential_evolution
import numpy as np

def f(x):
return np.sin(x[0]) + np.cos(x[1])

bounds = [0, 6, -3, 3]
result = differential_evolution(f, bounds)
print(result.x)

D. from scipy.optimize import differential_evolution
import numpy as np

def f(x):
return np.sin(x[0]) + np.cos(x[1])

bounds = [(0, 6), (-3, 3)]
result = differential_evolution(f)
print(result.x)

Solution

1. Step 1: Check function definition

   The function f correctly uses array indexing for x[0] and x[1] and applies numpy sine and cosine functions.

2. Step 2: Verify bounds and function call

   Bounds are correctly given as a list of tuples for each variable. The differential_evolution call includes both the function and bounds.

3. Final Answer:

   Code snippet C correctly defines the function, bounds, and calls differential_evolution -> Option A

4. Quick Check:

   Correct function and bounds usage = from scipy.optimize import differential_evolution import numpy as np def f(x): return np.sin(x[0]) + np.cos(x[1]) bounds = [(0, 6), (-3, 3)] result = differential_evolution(f, bounds) print(result.x) [OK]

Hint: Use list of tuples for bounds and array input function [OK]

Common Mistakes:

• Passing bounds as flat list instead of list of tuples
• Defining function without indexing array input
• Calling differential_evolution without bounds argument