End of semester project

Global Optimization algorithms

Ecole Polytechnique de l'Universit de Nice Sohia Antipolis
Applied Mathematics and Modelisation Department
June 2!2
"eacher#
Pierre D$E%&USS
Students#
A'(S")N) "ho
M*%AS SA$(U+(U ,assane
-,AN' .uan
!/20
Table des matires
)1 )ntroduction1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111112
))1 Simulated annealin3 al3orithm 4SA5111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111116
!1,istory111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111116
21Principle7al3orithm and choice o8 parameters111111111111111111111111111111111111111111111111111111111111111111111111111116
21Applications and results1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111119
)))1 Ant colony optimisation al3orithm 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!
!1,istory111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!
111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!
21Ant colony optimi:ation al3orithm 4A;(5 8or resolvin3 discrete pro<lem1111111111111111111111111111111!!
21! Principle7al3orithm and choice o8 parameters11111111111111111111111111111111111111111111111111111111111111111111!!
212 "he condition to stop1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!2
111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!2
212 Applications and results111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!2
21Ant colony optimi:ation al3orithm 4A;(5 8or resolvin3 continuous 8unction =ith multiple
e>tremums11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!?
21! Principle7al3orithm and choice o8 parameters11111111111111111111111111111111111111111111111111111111111111111111!?
212 @'al3orithm o8 reali:ation1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!?
212 Applications and results111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!6
)A1Particle s=arm optimisation al3orithm111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!B
!1,istory111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111!B
21Principle7al3orithm and choice o8 parameters111111111111111111111111111111111111111111111111111111111111111111111111111!B
21Applications and results111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111122
A1 ;omparison o8 al3orithms111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111112C
!1;alculation time11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111112C
21Num<er o8 iterations1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111112C
21 Precision11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111120
?1Speci8ities1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111120
2/20
I. Introduction

(ptimi:ation aims73enerally7 to 8ind the <est solution called optimum o8 a pro<lem <y usin3 a set o8
numeric methods1 )n this case7 =e are interested in al3orithms solvin3 optimi:ation pro<lems 8or
real7 continuous7 di88erentia<le and nonDlinear 8unctions1
Several approachs are availa<le7 there are local methods 3ivin3 a local optimum and 3lo<al ones
permitin3 to 8ind a 3lo<al optimum1 ,ere7 =e =ill seeE 8or minima <ecause a ma>im:ation pro<lem
can <e considered as a minimi:ation question1
Fe =ill study 3lo<al optimi:ation al3orithms7 in particular# simulated annealin37 ant colony and
particle s=arm1

2/20
)n the 8ollo=in37 =e search a 3lo<al minimum o8 these 8unctions#
J
!7
(x)=2xsin( x)
7 x! ; 2| and u=27 ?7 111 , ! 1
J
2
(x)=!sin(72 xsin(!72x +7! x
?
+72 x+9)) =ith x!; !| 1
'raphs#
?/20
C/20
0/20
II. Simulated annealing algorithm (SA)
1.History
"he method o8 annealin3 comes 8rom the o<servation that the natural coolin3 o8 certain metals does
not allo= atoms to move into the con8i3uration stron3est1
"he most sta<le con8i3uration is reached <y controllin3 the coolin3 and slo=in3 it do=n =ith a
supply o8 e>ternal heat1
"his process is used in metallur3y to improve the quality o8 a solid7 =e <rin3 ener3y to 8use the
metal1 "hen7 <y coolin3 the metal =ith the =ay more or less rapid7 =e seeE 8or reachin3 a minimal
ener3y state 4=hich corresponds to a solid structure51 Fe seeE the most sta<le structure possi<le 8or
this metal1
"he annealin3 method =as simulated 4<y Metropolis5 in the Cs7 the main idea is to move 8or
8indin3 the 3lo<al minimum <y choosin3 a nei3h<our point randomly1 "hen7 i8 the value o8 the
8unction is smaller at the nei3h<our point 4at the current temperature57 =e =ill move there1
(ther=ise7 =e =ill move there =ith a certain pro<a<ility 4=hich decreases the value o8 the 8unction
=hich si3n temperature51 &inally7 =e =ill repeate this process <y decreasin3 the temperature1
&urthermore7 this al3orithm =as used 8or solvin3 com<inatorial optimi:ation pro<lems in the 9s
4<y +ricEpatricE and ;erny5# Suppose a company =ants to distri<ute N di88erent tasEs 48or e>ample
related to the manu8acture7 receipt o8 components7 dispatchin3 o8 products7 accountin37 etc115 in N
local <uildin31 Each division has its advanta3es and disadvanta3es <y re3ardin3 the match <et=een
the tasE and the local1 Fe 8urther assume that =e can quanti8y7 or at least classi8y the e88ectiveness
o8 each =ay o8 the distri<utions in terms o8 cost o8 manu8actured 3oods 8or e>ample1 "he pro<lem is
to determine the most e88ective allocation =ithout tryin3 them allG
2.Principle,algorithm and choice of parameters
Principe #
6/20
&irst7 taEe a startin3 point7 initiali:e the ener3y and the temperature1
Ne>t7 at each iteration7 =e choose randomly a point close to the 8irst one and compute the ener3y at
this point1 )8 the ener3y at this point is lo=er than <e8ore7 =e apply it to the current solution1
(ther=ise7 it can <e accepted or reHected accordin3 to the Metropolis rule1
&inally7 temperature is decreased and process is repeated a3ain until ma>imum num<er o8 iterations
is reached1
IuasiDcode o8 the al3orithm#
s #J s
e #J E4s5
"J"ma>
n)ter #J
=hile n)ter K n)terMa> and e L eMa>
sn #J nei3h<our4s5
en #J E4sn5
i8 en K e or Metropolis$ule then
s #J snM e #J en
n)ter #J n)ter N !
"Jreduction4"5
return s
Metropolis $ule #De8ine the pro<a<ility that accept a nonDoptimal solution 4hopin3 to mana3e to a
lo=er minimum5 #
e
A E
T 1
"he 8act that unminimi:in3 solution may <e chosen 3uarantee not to 8all in local minimum1
"emperature $eduction #
"=o main =ays #
)terate =ith constant temperature until ener3y seems to <e in a local minimum1 "hen7 decrease
temperature1
Decrease continuously temperature =ith a continuous rule7 8or e>ample
ne="emperatureJ1BBcurrent"emperature1
"he =ay to reduce temperature is empirical choice1)ndeed7 =e could 8ind a linE <et=een the
movement num<er7 interval si:e and movement si:e1 ,o=ever7 tests sho= that a slo= continuous
reduction is e88ective i8 the 8unction doesn't 8luctuate too much1
3.Applications and results
!DD8unctions #
)n a space o8 dimension !
9/20
&unction J!72#
J
!7 u
( x)=2x sin(u x)
"he al3orithm conver3es in 2C iterations 7the computin3 time is 1!seconds and the minimum
3lo<al 8ound is >J!7291
&unction J2# J
2
(x)=!sin(72 xsin(!72x +7! x
?
+72 x+9))
"he al3orithm conver3es in 2C iterations 7the computin3 time is 7!seconds and the minimum
3lo<al 8ound is is 8ound is >J B191
B/20
III. Ant colony optimisation algorithm
1.History
Proposed <y Marco Dori3o in the BS7 the al3orithm =as inspired <y the comportement o8 a 3roup
o8 ants 8or 8indin3 the <est =ay <et=een the ants and the 8ood1
"his method aims to construct the <est solutions1 Every time an individual 8ind a solution o8 the
pro<lme7 =hatever 3ood or <ad7 it =ill enrich the collective Eno=led3e o8 the 3roup1 &urthermore7
every time a ne= individual chosse the =ay7 it =ill rely on the collective Eno=led3e to maEe the
choice1
*io3rapher#
Marco Dori3o 4<orn 20 Au3ust !B0! in Milan7 )taly5 is a research director 8or the *el3ian &unds 8or
Scienti8ic $esearch 4&N$S5 and a coDdirector o8 )$D)A7 the arti8icial intelli3ence la< o8 the
Universit li<re de *ru>elles1 ,e is the proponent o8 the Oant colony optimi:ationO metaheuristic
and one o8 the 8ounders o8 the s=arm intelli3ence research1
!/20
2.Ant colony optimization algorithm (ACO for resol!ing discrete
pro"lem
,ere7 =e taEe the @e pro<lPme NP complet 4voya3eur de commerce5 8or an e>ample7 the other
discrete pro<lems have e>actly the same principle1 "he pro<lem is that We have n cities, we want to
travel all of them following a shortest path.
2.1 Principle,algorithm and choice of parameters

The mechanism of this algorithm comes from the behavior of the ant colony
Step 1 -Initialize the distribution of the ant colony
=e put m ants in the n cities randomly1
Step 2 -Determine the rule of moe of the ant colony
"he E
th
ant in the city i =ill move to the ne>t city =ith the pro<a<ility as <elo=#

t (i , j ) si3ns the density o8 pheromone 4the marEer o8 ants7 more ants pass a path7 more
pheromone they =ill leave51
p(i , j )=!/ d (i , j ) is the in8ormation o8 inspiration =ith d4i7 H5 is the distance <et=een city i
and city H1
o and re8lect the the relative importance o8 the pheromone and the in8ormation o8
inspiration1

tabu
k
is the list o8 cities =hich the ants have already visited1
!!/20
Q 4 7 5R Q 4 7 5R
7
Q 4 7 5R Q 4 7 5R
4 7 5 4!5
7
k
k
k
s tabu
i j i j
if j tabu
i s i s
P i j
otherwise

Fhen all the ants have 8inished this travel round 4this iteration57 the pheromone =ill <e updated
as 8ollo=s#
p is the parameter =hich indicates the persistence o8 pheromone7 p! 1
is the quantity o8 pheromone =hich the ant
a
k
leaves =hen it moves 8rom city i to city H1 I is a
constant and
l
k
is the path o8 ant
a
k
in this iteration7

k
is its len3th1
2.2 #he condition to stop
!"# fix a number of times maximal of the iteration$
!%#the current solution dosn&t change for successive k times, which means that the algorithm has
already converged$

!2/20
5 2 4
5 4 5 4
!

=
=
+ = +
m
k
k
ij ij
ij ij ij
t n t


425

k
k
k ij
'
ij l

otherwise

2.3 Applications and results

An example for test: we have 23 cities which distribute randomly as follows:
(e )ut the coordinates of these %* cities in the matrix +, in this case,
+,-",*;%,.;/,0;"1,"";%",2;.,3;%*,.;*",**;%.,21;"%,0;%3,4;"0,20;.",.1;..,"/;*.,*1;2%,23;2.,%*;
2",%/;"/,*1;"0,2%;"4,%";*.,2/;24,"05 and the indice of city is -",%,$$$,%*5
(hich means the coordinate of city " is !",*#, and that of city % is !%,.#, etc$
(e solve the )roblem with the ant colony algorithm in matlab, after %*1 iterations, the solution
converge and we obtain the result as below6
The shortest )ath is
7hortest89oute ,
-"% %1 / "/ %" 2 3 % " * "1 . 4 "" %* "2 "4 "0 ". 0
"3 "* %%5
and the shortest length is %14$4**0$
!2/20
3.Ant colony optimization algorithm (ACO for resol!ing continuous
function $ith multiple e%tremums
3.1 Principle,algorithm and choice of parameters
!rinciple" Ant colony optimi:ation al3orithm 4A;(5 8or resolvin3 continuous 8unction =ith
multiple e>tremums
"he <ehavior o8 a sin3le individual o8 ants is very simple7 ho=ever a8ter 8ormin3 a 3roup7 they can
sho= an ama:in3 or3ani:ational <ehavior1 Ants have a very important 8eature7 that is in the process
o8 searchin3 8or 8ood7 they can al=ays 8ind the shortest path leadin3 <et=een the source o8 8ood and
their nests1 "he ant colony optimi:ation al3orithm is <ased on the <ehavior o8 ants searchin3 8or
8ood in nature1
Fe taEe the one dimension 8unction as an e>ample and =e search 8or its minimums1 Suppose the
8unction yJ84>5 is de8ined in interval Qa7 <R1 &irstly7 =e divide the interval into n small su<interval
=ith the same si:e7 marEed as
:
!
, :
2
,111 , :
n
an the midpoint o8
:
i
is noted as
x
i
1Assumin3 that the nei3h<our intervals o8
:
i
are
:
i!
and
:
i+!
7 and suppose that there is an
virtual ed3e
e( :
i
, :
i+!
)
<et=een
:
i
and
:
i+!
7 the =ei3ht o8 this ed3e 4the virtual distance 5
is related to the value o8
f ( x
i
) f ( x
i+!
)
7 the <i33er the value o8
f ( x
i
)f ( x
i+!
)
7 the more
possi<le that the ants =ill move 8rom the interval
:
i
to the interval
:
i+!
1 A8ter the ants movin3
to the interval
:
i+!
7 they =ill leave the pheromone7 the quantity o8 the pheromone is also related
to
f ( x
i
)f ( x
i+!
)
7 the more pheromone in the interval
:
i+!
7 the more possi<le it =ill appeal
to the ants to come1
A8ter the ants movin37 some o8 the intervals have many ants7 the others have not any ant1 "he
intervals =hich contain the ants are e>actly =here the e>tremums are1 (n the other hand7 the
intervals =hich have no ant =ill almost impossi<le contain the e>tremums1 Fe taEe out the intervals
=hich contain the ants and =e re8ine these intervals a3ain7 and =e repeat the search process a<oveD
mentioned until the re8ined intervals are enou3h small1 &inally7 all the ants stop at the points o8
e>tremums or near<y7 the midpoints o8 the intervals =here the ants stays are the e>tremums1
"o sum up7 the reali:ation o8 the ant colony optimi:ation al3orithm consist o8 the ? 8ollo=in3 steps#
Step ! D)nitiali:e the distri<ution o8 the ant colony
Step 2 DDetermine the rule o8 move o8 the ant colony
Step 2 DUpdate o8 the pheromone
Step ? DNarro= the research space
3.2 &'algorithm of realization
Step 1 -Initialize the distribution of the ant colony
!?/20
=e divide equally Qa7 <R into n su<interval7 the len3th o8 each su<interval equals
6=
ba
n
7 each
su<interval is noted as
:
!
, :
2
,111 , :
n
7 the i
th
su<interval :
i
= a+(i !)6 , a+i 6| and its
midpoint
x
i
=a+(i
!
2
)6
1 At the very <e3innin3 moment7 =e put an ant at the midpoint o8 each
su<interval and they are noterd respectively as
a
!
, a
2
,111 , a
n
7 each ant
a
i
has a 8unction value
f ( x
i
)
1 Fe note also
t
i
(t )
as the pheromone o8 su<interval
:
i
at the moment t1 At the 8irst
moment7 each su<interval has the same quantity o8 pheromone7 =hich means
t
i
()=const
4const
is a positive constant57 and =e initiali:e also the increment o8 pheromone At
i
=7 (i =!7111 , n) 1
'raph !# the initial distri<ution o8 the ant colony1
Step 2 -Determine the rule of moe of the ant colony
Fe note
neighbor
(
:
i
) as the set o8 nei3h<orhood o8
:
i
7 8or the one dimension 8unction =e
have then #
neighbor ( :
i
)=

:
i +!
i =!
:
i!
, :
i+!
i=27111 , n!
:
i !
i=n
1 "he ant at the su<interval
:
i =ill move
to its nei3h<orhood accordin3 to the =ei3ht o8 virtual ed3e
e( :
i
, :
i +!
)
#
f ( x
i
) f ( x
i+!
)
7 =e
de8ine the 8unction o8 inspiration
p
ij
=

f ( x
i
)f ( x
i+!
)
1 Fe suppose that the ant
a
k
=as
currently at the su<interval
:
i
7 i8
f ( x
i
) f ( x
i+!
)>
7 the ant
a
k
=ill move to its nei3h<or
:
j
M other=ise7 it =ill not move1 Fe note
allowed
k
as the set o8 su<intervals =here the ant
a
k
can move to ne>t step and )
ij
(k )
(t ) as the pro<a<ility =ith =hich the ant
a
k
move 8rom
su<interval
:
i
to
:
j
at the t
th
iteration1 Fe have the e>pression o8 )
ij
(k )
(t ) as 8ollo=s#

!C/20
Y
x
x1 x2 x3 x4 x5 x6 x7 x8 x10
f(x1)
f(x2)
f(x3) f(x4)
f(x5)
f(x6)
f(x7)
f(x8)
f(x9)
0
)
ij
(k )
(t )=

t
j
o
(t ) p
ij

hallowed
k
t
h
o
(t )p
ih

j allowed
k
otherwise
o is the 8actor si3nin3 the inspiration o8 in8ormation and beta is the 8actor si3nin3 the
inspiration o8 e>pectation1
Step # -$pdate of the pheromone
)8 the ant
a
k
move 8rom su<interval
:
i
to
:
j
accordin3 to the pro<ility7 it =ill leave the
pheromone in the su<interval
:
j
7 =e note the quantity o8 pheromone it leaves as At
j
( k)
(t ) 7 so
=e have At
j
( k)
(t )=+
!
( f ( x
i
) f ( x
j
)) 7 the 3reater the value o8
f ( x
i
) f ( x
j
)
7 the more
pheromone the ant
a
k
=ill leave7 the more pro<a<le the su<interval
:
j
=ill appeal to other
ants1 Each ant =ho moves to
:
j
=ill all leave the pheromone7 in this case7 =e suppose that there
are q ants movin3 to
:
j
at the t
th
iteration7 so the total pheromone they leave is
At
j
(t )=

)=!
;
At
j
( j
)
)
(t ) 1 A8ter all the ants 8inish this iteration o8 movement7 =e update the
pheromone as 8ollo=s#
t
j
(t +!)=(!p)t
j
(t )+At
j
(t )
1 )n =hich7 p is the coe88icient
si3nin3 the volatilisation o8 pheromone and !p is the coe88icient si3nin3 the remainder o8
pheromone1
"his mechanism is viey reasona<le and e88ective <eacause i8 many ants assem<le to one su<interval
=hich means the value o8 8unction at this point is very small7 the pheromone in this su<interval =ill
<e very much and the 3reat quantity o8 pheromone =ill appeal to more ants to come7 =e =ill 8inally
8ind this point7 it may <e the minimum1 (ther=ise7 the su<interval =ith 3reat 8unction value =ill <e
8or3otten <ecause less ants pass it and its pheromone =ill volatili:e7 thus it =ill not appeal to the
ants to come1
Step % -&arro' the research spac e
A8ter some iterations7 =hen all the ants stop movin37 the distri<ution o8 intervals =ill have the
8eature as 8ollo=s# the su<intervals =hich contan the ants =ill must contain the minimaM the
su<intervals =hich have no ant =ill impossi<ly have the minima1 Fe taEe out these su<intervals and
re8ine them# at the ne>t iteration =e maEe the ants search 8or the minima only in these su<intervals1
A8ter many iterations7 the scope o8 searchin3 =ill <e smaller and smaller and =hen the su<interval
re8ined is enou3h small7 all the ants =ill stop near<y the minima1
!0/20
3.3 Applications and results
&unction J
2
(x)=!sin(72 xsin(!72x +7! x
?
+72 x+9))
Fith the ant colony al3orithm7 =e 8ind the 3lo<al minimum is 619!2 4the point in cercle5 =ith the
error J 1! in 9 iterations1
!6/20
&unction
J
!72
( x)=2xsin (2x)
Fith the ant colony al3orithm7 Fe 8ind the 3lo<al minimum is !126 4the point in cercle5 =ith the
error J 1! in 6 iterations1
'raph2# "he relation <et=een compute time and the num<er o8 ants#
!9/20
IV.Particle sarm optimisation algorithm
1.History
"his al3orithm has <een inspired <y the livin3 =orld1 )t's <ased on a model developed <y ;rai3
$eynolds at the end o8 the 9's7ena<lin3 to simulate the movement o8 a 3roup <irds or 8ishes1 An
other source o8 inspiration 7de8ended <y some people7<elon3 to socialDpsycholo3y1
"his optimi:ation method is <ased on mutual colla<oration o8 a3ents1)t's similar to ant colony
al3orithms7=hich also rely on a autoDor3ani:ation concept1"his idea supports a 3roup o8
unintelli3ent individuals can have a comple> 3lo<al or3ani:ation1"hat means each a3ent moves
accordin3 to its <est past per8omance and the <est per8omance o8 its nei3h<ourhood1
2.Principle,algorithm and choice of parameters
Fe have N points
)
i
randomly distri<uted <y an uni8orm pro<a<ility la= over a part o8 the
space 3iven <y a So<ol sequence1
"he 3oal o8 the al3orithm is to assem<le these points7 that =e called hence8orth particles7 as much
as possi<le to the 3lo<al minimum o8 the 8unction that =e seeE to optimi:e1 "he set o8 particles is
called s=arm1
&or each particle
)
i
7 =e de8ine its nei3h<ourhood
<
i
8ormed <y the set o8 particles
)
j
=hich are OcloseO to
)
i
1"he notion o8 closeness can <e considered 8rom several di88erent
=ays1Each o8 them creates a particular particules 's topolo3y 1
E>amples o8 topolo3y #
)n star # each particle is linEed to all the others1
)n radius # Particles are connected to an only one central particle1
"here are others Einds o8 topolo3ies 1
Al3orithm #
"he <ases o8 this al3orithm are the 8ollo=in3 #
DEach particle o8 the s=arm have a memory =hich retain the <est solution o8 the optimi:ation
pro<lem it has achieved so 8ar1
DEach particle is ena<le to communicate =ith the particles o8 its nei3h<ourhood1
DEach particle move in the space <y 8ollo=in3 a trend =hich depends on its intention to 3o to the
<est solution it has 8ound 4noted
x)=est
i
5 and that to =ant to 3o to the optimal solution o8 his
nei3h<ourhood1
Fe intend to solve the 8ollo=in3 optimisation pro<lem #

Trouver xD
f ( x)=min
x D
f ( x)
x=argmin
x D
f ( x)
1
Step ! # ;reate a set o8 points4 particles5 uni8ormly distri<uted over D 1
!B/20
Step 2# Evaluate the 8unction 8 in each position
x
i
correspondin3 to particle
)
i
1
Step 2 # &or each particle7i8 the current position
x
i
is <etter than the <est solution Eno=n <y7 ie7
i8
f (x
i
(t )) f ( x)=est
i
(t !))
7 then the value o8 the <est solution Eno=n <y
)
i
is updated as
8ollo=s #
x)=est
i
(t )=x
i
(t )
1
Step ? # Determine 8or each particle the position o8 the <est solution o<tained <y the nei3h<ourhood
xv=est
i
(t!)
1i8 the current position is <etter7 ie7 i8
f ( x
i
(t )) f ( xv=est
i
(t !))
7then the
value o8 the <est solution Eno=n <y the nei3h<ourhood is updated as 8olo=s #
x)=est
i
(t )=x
i
(t )
Step C # "he movin3 o8 each particle is computed taEin3 into account a conseravative trend 48ollo=
its o=n solution5 and a trend to 8ollo= the nei3h<ourhood's solution1 "he movin3 is called velocity
o8 the particle1)t's de8ined <y #
v
i
(t+!)=v
i
(t )+c
!
r
!
(t )(x)=est
i
(t )x
i
(t ))+c
2
r
2
(xv=est
i
(t )x
i
(t ))
=ith
c
!
>
7
c
2
>
et
c
!
+c
2
?
and called inertia 8actor1
r
!
et
r
2
are random varia<les =hich 8ollo= a uni8orm la= over ; !| 1
Step 0 #
"he particles
)
i
are moved to their ne= position <y #
x
i
(t +!)=x
i
(t )+v
i
( t+!)
Step 6 #
'o <acE to step 2 until one o8 stoppin3 criteria are satis8ied1
"he 8ollo=in3 8i3ures 3ive an idea o8 the particle s=arm optimi:ation al3orithm =ith a 2DD8unction#
J (x , y)=!( x
2
y)
2
+( x!)
2
$osen<rocE's O *anana 8unctionO 4see pa3e !C51
2/20
2!/20
22/20
;hoice o8 parameters
"he particule s=arm optimi:ation al3orithm depends on some parameters =hich7accordin3 to the
pro<lem7 must <e chosen care8ully1 )tSs num<er o8 particles N7 topolo3y and the si:e o8
nei3h<ourhood7 the inertia 8actor =hich impose a ma>imal velocity7the coe88icients
c
!
and
c
2
1
"he set o8 these parameters determine the conver3ence o8 the method and a<ove all his speed o8
conver3ence1
"he more particules there are7 the 8aster a 3lo<al minimum =ill <e approached 1 "he topolo3y is
also very important1&or e>ample7topolo3y in star 3uarantee to approach all particles to 3lo<al
minimum M topolo3y in star =ould permit to approch central particule to 3lo<al minimum1
"he inertia 8actor represents the a<ility to e>plore o8 each particule1 A 3reat value o8 4L !5 is
synonym o8 a lar3e ran3e o8 motion7 there8ore o8 3lo<al e>ploration1 At the opposite a small value
o8 4K !5 is synonym o8 a small ran3e o8 motion7 and there8ore o8 local e>ploration1
;oe88icients
c
!
et
c
2
characteri:e respectively conservative trend and trend to 8ollo= the
nei3h<ourhood1 )8 the 8irst is 3reater than the other7a particle =ould try to approach its <est solution
more than that o8 his nei3h<ourhood and inversely1 "he t=o parameters must <e chosen also
relyin3 on the si:e o8 space to e>plore1 &or a continuous 8onction =hich o8ten varies 7thus =hich has
numerous e>trema7 they must <e taEen enou3h small1
3.Applications and results
)n a space o8 dimension !
&unction J!72#
J
!7 u
( x)=2x sin(u x)
"he al3orithm conver3es in 26 iterations 7the computin3 time is 1B0?62 seconds and the
minimum 3lo<al 8ound is >J!120B9 1
&unction J2# J
2
(x)=!sin(72 xsin(!72x +7! x
?
+72 x+9))
22/20
"he al3orithm conver3es in 2C iterations 7the computin3 time is 91!9229 seconds and the
minimum 3lo<al 8ound is >JD619!221
)n a space o8 dimension 2
&unction# J (x , y)=!( x
2
y)
2
+( x!)
2
'raph# $osen<rocE's O *anana 8unctionO
"he al3orithm conver3es in 22 iterations 7the computin3 time is 1C!!?0 and the minimum 3lo<al
8ound is Q!1!! 7 !122R 1
)n a space o8 dimension 2
&unction J# J (x , y , >)=(x!)
2
+( y!)
?
+(>+!!)
2
C;
"he al3orithm conver3es in 22 iterations 7the computin3 time is 1C!!?0? seconds and the
minimum 3lo<al 8ound is >JQD10BB9 7 212!?C 7 D!10B?R
)n a space o8 dimension n
&unction J#
J
n
(x)=
!
2
x
T
?
n
xb
n
x
=ith ?
n
=
(
? 2 111
2 ? 2 111
2 ? 111

111 2 ?
)
a 4n7n5Dmatri> and
b
n
=!7!1117!|
a nDsi:e vector1
Fith nJ27 the al3orithm conver3es in 90 iterations 7the computin3 time is !100!B0 seconds and
the minimum 3lo<al 8ound is >JQ16?B9 7 1BBBC 7 16?B!R 1
Fith nJ!7 the al3orithm conver3es in 96? iterations7the computin3 time is !!1?B0!B0 seconds and
the minimum 3lo<al 8ound is#
>JQ219?20 7 ?12692 7 C1!!90 7 C10C6B 7 01!0B! 7012B6 7C1B22 7C12BC2 7C1!26C 7212022R
2?/20
Fith nJ!7 the al3orithm conver3es in C2!2 iterations 7the computin3 time is !CC1!B69CB seconds
and the minimum 3lo<al 8ound is a vector o8 ! components1
V. !omparison o" algorithms
1.Calculation time
&unction J!72#
Optimisation algorithm (alculation time
Simulated annealin3 1!CC?? seconds
Ant ;olony 1026?? seconds
Particle s=arm 1B0?62 seconds
&unction J2#
Optimisation algorithm (alculation time
Simulated annealin3 10?0! seconds
Ant ;olony 10C602 seconds
Particle s=arm 91!9229 seconds
;omments#"he simulatin3 annealin3 is the 8astest1
2.(um"er of iterations
&unction J!72#
Optimisation algorithm &umber of iterations
Simulated annealin3 2
Ant ;olony 0
Particle s=arm 26
&unction J2#
Optimisation algorithm &umber of iterations
Simulated annealin3 2
Ant ;olony 6
Particle s=arm 2C
;omments# "he num<er o8 iterations o8 ant colony al3orithm is the least o8 the three1"he others
need very more iterations1
2C/20
3. Precision
&unction J!72#
Optimisation algorithm Error )!recision*
Simulated annealin3 !TD2
Ant ;olony !TD2
Particle s=arm !TD2
&unction J2#
Optimisation algorithm Error)!recision*
Simulated annealin3 !
Ant ;olony 21!TD!
Particle s=arm !TD2
;omments# Ant colony and particle s=arm 3ives a lo=er error than the simulated annealin31
).*pecifities
"he simulated annnealin3 is 8aster than the t=o other <ut less precise and i8 it conver3es7it can <e
need many iterations to conver3e to a 3lo<al minimum1
"he ant colony al3orithm is 8aster than the particle s=arm <ut it is not simple to pro3ram 8or
8unction =hich have more than 2 varia<les1 )n the case o8 oneDdimension 8unction it's as precise as
the particle s=arm1
"he particule s=arm has the main advanta3e to <e usa<le 8or a 8unction o8 any dimension and its
precision can <e chosen or controlled1
!onclusion
"o sum up7 =e have tested three al3orihms 8or continuous optimi:ation# 8ind a 3lo<al
minimum 8or real7 continuous7 di88erentia<le and nonDlinear 8unctions1Each al3orithm has its o=n
advanta3e7 ho=ever since they all aim to 8ind the 3lo<al optimum7 so they taEe a little time than
classical al3orithms 3ivin3 local minimum1
20/20