Simulated annealing (dual_annealing) in SciPy - Step-by-Step Execution

Concept Flow - Simulated annealing (dual_annealing)

Start with initial guess

↓

Evaluate objective function

↓

Generate new candidate solution

↓

Calculate energy difference

↓

Accept new solution?

↓

Update current

↓

Lower temperature

↓

Temperature > min?

↓

Stop

Yes↓

Repeat steps

The algorithm starts with a guess, evaluates it, tries a new guess, decides to accept it or not based on energy difference and temperature, then lowers temperature and repeats until stopping.

Execution Sample

SciPy

from scipy.optimize import dual_annealing

def f(x):
    return (x[0]-3)**2 + (x[1]+1)**2

result = dual_annealing(f, bounds=[(-5,5), (-5,5)])
print(result.x, result.fun)

This code finds the minimum of a simple function using dual_annealing from scipy.

Execution Table

Step	Current Solution	Candidate Solution	Energy Current	Energy Candidate	Energy Diff	Temperature	Accept Candidate	Reason
1	[0, 0]	[0.5, -0.3]	10	8.34	-1.66	2500	Yes	Candidate better (energy lower)
2	[0.5, -0.3]	[1.2, -0.8]	8.34	4.13	-4.21	2400	Yes	Candidate better
3	[1.2, -0.8]	[2.5, -1.5]	4.13	0.5	-3.63	2300	Yes	Candidate better
4	[2.5, -1.5]	[3.5, -0.5]	0.5	1.25	0.75	2200	No	Candidate worse, rejected
5	[2.5, -1.5]	[3.1, -1.1]	0.5	0.02	-0.48	2100	Yes	Candidate better
6	[3.1, -1.1]	[2.9, -0.9]	0.02	0.08	0.06	2000	Yes	Candidate worse but accepted by probability
...	...	...	...	...	...	...	...	...
N	[3.0, -1.0]	[3.0, -1.0]	0.0	0.0	0.0	1	Yes	Reached minimum, temperature low
Exit	Final solution found		0.0			0	Stop	Temperature reached minimum threshold

💡 Temperature reached minimum threshold, algorithm stops with best found solution.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	After 5	After 6	...	Final
Current Solution	[0, 0]	[0.5, -0.3]	[1.2, -0.8]	[2.5, -1.5]	[2.5, -1.5]	[3.1, -1.1]	[2.9, -0.9]	...	[3.0, -1.0]
Temperature	2500	2400	2300	2200	2100	2000	...	...	0

Key Moments - 3 Insights

Why does the algorithm sometimes accept a worse candidate solution?

Because of the temperature and energy difference, the algorithm accepts worse solutions with some probability to avoid local minima, as shown in step 6 where energy difference is positive but candidate is accepted.

What causes the algorithm to stop?

The algorithm stops when the temperature reaches a minimum threshold, as shown in the exit row where temperature is 0 and no further steps occur.

Why does the current solution sometimes stay the same after a candidate is rejected?

When the candidate solution has higher energy and is not accepted, the current solution remains unchanged, as in step 4 where candidate is worse and rejected.

Visual Quiz - 3 Questions

Look at the execution table at step 4. What is the reason the candidate solution was rejected?

A Candidate solution had lower energy

B Candidate solution had higher energy and was rejected

C Temperature was too high

D Algorithm reached stopping condition

💡 Hint

Check the 'Energy Diff' and 'Accept Candidate' columns at step 4 in the execution table.

According to the variable tracker, what is the temperature after step 3?

A 2100

B 2200

C 2300

D 2400

💡 Hint

Look at the 'Temperature' row and the 'After 3' column in the variable tracker.

If the temperature never decreased, how would the acceptance of worse candidates change?

A Worse candidates would be accepted more often

B Worse candidates would never be accepted

C Acceptance would be random

D Algorithm would stop immediately

💡 Hint

Recall that higher temperature increases probability to accept worse solutions, see step 6 in execution table.

Concept Snapshot

Simulated annealing (dual_annealing):
- Starts with initial guess and temperature
- Generates new candidate solutions
- Accepts better or sometimes worse solutions based on temperature
- Gradually lowers temperature
- Stops when temperature is low
- Helps find global minimum by escaping local minima

Full Transcript

Simulated annealing with dual_annealing starts by picking a guess and temperature. It checks the function value (energy). Then it tries a new guess nearby. If the new guess is better, it accepts it. If worse, it might still accept it depending on temperature. Temperature lowers step by step. This repeats until temperature is very low. The final guess is the best found minimum. This method helps avoid getting stuck in bad local minimum by sometimes accepting worse guesses early on.

Practice

1. What is the main purpose of using dual_annealing in scipy.optimize?

easy

A. To find the minimum value of a function within given bounds

B. To sort a list of numbers in ascending order

C. To calculate the mean of a dataset

D. To generate random numbers following a normal distribution

Solution

1. Step 1: Understand the purpose of dual_annealing

   dual_annealing is an optimization method used to find the minimum of a function, especially when the function is complex and has many local minima.

2. Step 2: Identify the correct use case

   Among the options, only finding the minimum value of a function within bounds matches the purpose of dual_annealing.

3. Final Answer:

   To find the minimum value of a function within given bounds -> Option A

4. Quick Check:

   Optimization = Find minimum [OK]

Hint: dual_annealing is for minimizing functions with bounds [OK]

Common Mistakes:

• Confusing optimization with sorting or statistics
• Thinking dual_annealing generates random numbers
• Assuming it calculates averages

2. Which of the following is the correct way to import dual_annealing from scipy.optimize?

easy

A. from scipy.optimize import dual_annealing

B. import dual_annealing from scipy.optimize

C. from scipy import dual_annealing.optimize

D. import scipy.optimize.dual_annealing

Solution

1. Step 1: Recall Python import syntax

   The correct syntax to import a function from a module is from module import function.

2. Step 2: Match syntax to options

   from scipy.optimize import dual_annealing matches the correct syntax: from scipy.optimize import dual_annealing. Other options have incorrect syntax.

3. Final Answer:

   from scipy.optimize import dual_annealing -> Option A

4. Quick Check:

   Correct import syntax = from scipy.optimize import dual_annealing [OK]

Hint: Use 'from module import function' to import dual_annealing [OK]

Common Mistakes:

• Using 'import function from module' which is invalid
• Trying to import submodules incorrectly
• Using dot notation in import statements wrongly

3. What will be the output of the following code snippet?

from scipy.optimize import dual_annealing

def f(x):
    return (x[0] - 3)**2 + (x[1] + 1)**2

bounds = [(-5, 5), (-5, 5)]
result = dual_annealing(f, bounds)
print(round(result.fun, 2))

medium

A. 10.00

B. 0.00

C. 4.00

D. Error

Solution

1. Step 1: Understand the function and bounds

   The function f(x) calculates the sum of squares of (x[0]-3) and (x[1]+1). The minimum is at x[0]=3 and x[1]=-1, where the function value is 0.

2. Step 2: dual_annealing finds the minimum within bounds

   The bounds allow x[0]=3 and x[1]=-1. So the optimizer should find the minimum function value close to 0. The print statement rounds the result to 2 decimals.

3. Final Answer:

   0.00 -> Option B

4. Quick Check:

   Minimum value = 0.00 [OK]

Hint: Minimum of squared distance function is zero at target point [OK]

Common Mistakes:

• Assuming the minimum is outside bounds
• Confusing function value with input values
• Expecting an error due to function shape

4. Identify the error in the following code using dual_annealing:

from scipy.optimize import dual_annealing

def f(x):
    return x**2

bounds = [(-2, 2)]
result = dual_annealing(f, bounds)
print(result.x)

medium

A. dual_annealing requires no bounds argument

B. Bounds should be a tuple, not a list

C. Function f expects a scalar but dual_annealing passes an array

D. Missing import for numpy

Solution

1. Step 1: Check function input type

   dual_annealing passes an array (even if one variable), but f(x) expects a scalar x. This mismatch causes an error.

2. Step 2: Verify bounds and imports

   Bounds as a list of tuples is correct. dual_annealing requires bounds. No numpy import needed here.

3. Final Answer:

   Function f expects a scalar but dual_annealing passes an array -> Option C

4. Quick Check:

   Function input type mismatch = Function f expects a scalar but dual_annealing passes an array [OK]

Hint: dual_annealing passes array input; function must accept array [OK]

Common Mistakes:

• Assuming bounds format is wrong
• Thinking numpy import is mandatory here
• Ignoring input type mismatch

5. You want to minimize the function f(x) = (x[0]-2)^2 + (x[1]-3)^2 but only allow x[0] between 0 and 1, and x[1] between 2 and 4. Which code correctly uses dual_annealing to find the minimum within these bounds?

hard

A. bounds = [(0, 1), (2, 4)]
result = dual_annealing(f)

B. bounds = [(2, 3), (3, 4)]
result = dual_annealing(f, bounds)

C. bounds = [(0, 2), (2, 3)]
result = dual_annealing(f, bounds)

D. bounds = [(0, 1), (2, 4)]
result = dual_annealing(f, bounds)

Solution

1. Step 1: Understand the function and bounds

   The function minimum is at x[0]=2, x[1]=3. But bounds restrict x[0] to [0,1] and x[1] to [2,4]. So the optimizer must search within these bounds.

2. Step 2: Check code options for correct bounds and usage

   The code bounds = [(0, 1), (2, 4)] result = dual_annealing(f, bounds) correctly sets bounds as [(0,1), (2,4)] and passes them to dual_annealing. The code bounds = [(2, 3), (3, 4)] result = dual_annealing(f, bounds) has wrong bounds. The code bounds = [(0, 2), (2, 3)] result = dual_annealing(f, bounds) has wrong bounds. The code bounds = [(0, 1), (2, 4)] result = dual_annealing(f) misses bounds argument.

3. Final Answer:

   bounds = [(0, 1), (2, 4)] result = dual_annealing(f, bounds) -> Option D

4. Quick Check:

   Correct bounds and function call = bounds = [(0, 1), (2, 4)] result = dual_annealing(f, bounds) [OK]

Hint: Bounds must match variable limits and be passed to dual_annealing [OK]

Common Mistakes:

• Using wrong bounds that exclude minimum
• Not passing bounds argument to dual_annealing
• Confusing variable order in bounds