EVOLUTIONARY PROGRAMMING WITH NON-CODING SEGMENTS

FOR REAL-VALUED FUNCTION OPTIMIZATION

Yoshiyuki Matsumura, Kazuhiro Ohkura and Kanji Ueda
Graduate S hool of S ien e and Te hnology, Kobe University,
Rokkodai, Nada, Kobe, 657-8501, Japan

fmatsumu,ohkura,uedagmi-2.me h.kobe-u.a .jp

ABSTRACT

Evolutionary Programming(EP) onstitutes a lass of

general optimization algorithms based on the model of
natural evolution. Real-valued fun tion optimization
has also been in luded in this domain sin e the early
90's. One of the main features is that the mutation size
is self-adaptively ontrolled by additional parameters
(strategy parameters). However, it was re ently found
that the mutation size ontrol is not \su iently" selfadaptive: EP often pra ti ally stops its geneti sear h
before rea hing a global optimum, due to the tenden y
that strategy parameters onverge in the early generations, with the result of that population annot move
to the other pla e. In this paper, in order to over ome this burdensome hara ter, an extended EP that
utilizes the geneti drift as another sour e of hanging strategy parameters is proposed. Computer simulations were ondu ted using several test fun tions
in order to evaluate the performan e of the proposed
method.
1

INTRODUCTION

Evolutionary Computation has been widely re ognized

as a robust approa h to various kind of optimization
problems. There are three main streams in this eld,
i.e., Evolution Strategies (ES), Geneti Algorithms
(GA) and Evolutionary Programming (EP)[1. EP was
rst proposed as an evolutionary approa h to nite
state ma hines in order to solve predi tion tasks, but
EP has been su essfully applied to real-valued fun tion optimization using normally distributed (Gaussian) mutation[3 sin e the early 90's.
Sin e then, the dis ussion on what type of mutation performs better has started. Yao and Liu[8 proposed to use Cau hy mutation instead of Gaussian mutation. They examined its performan e using 23 test
fun tions and found that Cau hy mutation is more effe tive for multimodal problems than onventional mutation, although lear dieren e was not observed for
unimodal problems. Inspired by this, several types of
the ombination of Chau hy and Gaussian have been
proposed[9, 2, 7.
On the other hand, another di ulty has been
found in the evolutionary dynami s. In onventional EP, mutation size is adjusted by its own \selfadaptive" property derived from strategy parameters

that roughly determine the mutation size. However,

EP still holds the hara ter of premature onvergen e,
i.e., EP often onverges before nding the global optimum even when the fun tion being solved is unimodal. This is aused by the tenden y that strategy parameters be ome nearly zeros in the early generations. The most popular s heme for avoiding this
undesirable behavior is to adjust strategy parameters
to be larger than a ertain small positive value (lower
bound). However, Liang, et al.[5 showed that the optimization performan e hanges largely with the lower
bound value and the optimal lower bound is problemdependent. They also proposed the dynami al ontrol of the lower bound as one way of over oming this
brittleness[6.
In this paper, another approa h to avoiding this
di ulty in EP is proposed. Our motivation is simple:
Sin e natural sele tion annot ontrol strategy parameters enough to ontinue geneti sear h, another sour e
for hanging the parameters an be added. We sele t
the geneti drift[4 for it. The geneti drift is referred to
as the phenomenon of sto hasti hange of inheritan e
where natural sele tion does not work. In order to
utilize this ee t, neutral region alled ina tive strategy parameters are inserted as non- oding segments in
the representation of an individual. These parameters
an a umulate various geneti hanges through generations using the ee t of geneti drift. Furthermore, asso iated with the representation, new mutation me hanisms are introdu ed so that onventional (a tive)
strategy parameters are sto hasti ally repla ed with
the inserted ina tive strategy parameters.
The proposed method is alled Robust-EP (REP)
in this paper, in the sense that REP is free from the
lower bound problem.
The rest of this paper is organized as follows: Se tion 2 overviews the omputational steps of EP. Se tion 3 des ribes the proposed approa h in detail. Se tion 4 shows the results of our omputer simulations.
Finally, Se tion 5 gives on lusions of this paper.
2

EVOLUTIONARY PROGRAMMING

The omputational steps of onventional EP is des ribed in this se tion. The notations in [8 are
adopted. For instan e, CEP and FEP stand for Fogel's EP and Yao and Liu's EP, respe tively.

2.1

Classi al Evolutionary Programming

The CEP employed in this paper is des ribed in the

following way. For i = 1; 2; : : : ;  and j = 1; 2; : : : ; n,
ea h individual Xi is represented as:
Xi
tx
i
t
i

= [xi ;  i
= (xi (1); : : : ; xi (j ); : : : ; xi (n))
= (i (1); : : : ; i (j ); : : : ; i (n))

(1)
(2)
(3)

where xi (j ); i (j ) 2 R, and xi (j ) and i (j ) denote the

omponent values of the ve tors xi , i , respe ~ i is al ulated using Gaussian
tively. An ospring X
mutation in the following way:

j -th

~i (j )
x
~i (j )

=
=

i (j ) exp( 0 N (0; 1) +  Nij (0; 1))

xi (j ) + ~i (j )Nij (0; 1)

(4)
(5)

where N (0; 1) denotes a real number taken from a normal distribution with its mean and standard deviation
are zero and one, respe tively. Nij (0; 1) is a random
number generated anew for every i and
j . The fa p p  1
0
tors  and  are ommonly set to
2 n
and

2n [1. The omputational pro edure of CEP an

be des ribed as follows:
1. Generate the initial population of
and set g = 1.

individuals,

2. Evaluate the obje tive value for ea h individual Xi ,

8i 2 f1; 2; : : : ; g of the population based on the
obje tive fun tion f (xi ).
3. Ea h parent Xi ,i = 1; 2; : : : ; , reprodu es ospring
~ i.
X
~ i , 8i 2
4. Cal ulate the tness of ea h ospring X
f1; 2; : : : ; g, a ording to f (x~i ).
5. Condu t pairwise omparison over the union of par~ i , 8i 2 f1; 2; : : : ; g. For
ents Xi and ospring X
ea h individual, q opponents are hosen randomly
from all the parents and ospring with an equal
probability. For ea h omparison, if the individual's tness is no smaller than the opponent's one,
it re eives a \win".
6. Sele t the  individuals out of parents Xi and o~ i , 8i 2 f1; 2; : : : ; g, having larger number
spring X
of \win", to make them parents of the next generation.
7. End the pro edure if the halting riterion is satised, otherwise, g = g + 1 and go to step 3.
2.2

Fast Evolutionary Programming

FEP is the form of EP in whi h Gaussian mutation is

repla ed with Cau hy mutation. FEP performs better
than CEP on multimodal fun tions, be ause of a larger

probability of es aping from lo al optima due to the

longer tails of the Cau hy mutation operator. FEP
implemented in this paper is exa tly the same as the
CEP des ribed in previous se tion ex ept for Eq.(5),
whi h is repla ed by as follows:
x
~i (j )

= xi (j ) + ~i (j )ij

(6)

where ij is a random number al ulated anew for ea h

and j based on the following Cau hy distribution
fun tion:
i

Ft (x)

= 1=2 + (1= ) ar tan(x=t)

(7)

where t = 1 in the this paper.

3

ROBUST EVOLUTIONARY
PROGRAMMING

Based on the motivation des ribed in Introdu tion, the

following individual representation Xi is adopted:
Xi
tx
i

t

ik

= [xi ; ( i0 ; : : : ;  ik ; : : : ;  im )
= (xi (1); : : : ; xi (j ); : : : ; xi (n))
= (ik (1); : : : ; ik (j ); : : : ; ik (n))

(8)
(9)
(10)

where i = 1; 2; : : : ; , j = 1; 2; : : : ; n and k =
0; 1; : : : ; m, respe tively. xi (j ); ik (j ) 2 R, and xi (j )
and ik (j ) denote the j -th omponent values of the
ve tors xi and ik , respe tively. Noti e that ea h xi (j )
has (m + 1) strategy parameters.
~ i is al ulated in the following
An ospring X
pro edure. Firstly, the omponent values ~ik are
al ulated by three sto hasti mutation operations
(Odup , Odel and Oinv ). For i = 1; 2; : : : ; , j =
1; 2; : : : ; n, k = 0; 1; : : : ; m, l 2 f1; 2; : : : ; mg and
l0 = 1; 2; : : : ; m(l0 6= l), they are dened as follows:
 0 i0 (j )

Odup

= i0 (j )

Odel

~ik (j ) = D( 0 ik (j ))
:  0 i(l 1) (j ) = il (j )

 0 il (j ) = i(l

 0 im (j )

Oinv

1)

(j )

= min(max ;

~ik (j ) = D( 0 ik (j ))
 0 i0 (j ) = il (j )
 0 il (j )

= i0 (j )
~i0 (j ) = D( 0 i0 (j ))
~il (j ) = D( 0 il (j ))
~il0 (j ) = il0 (j )

(11)

m
X1
k=0

ik (j ))

(12)

(13)

where max is a onstant. The three operations onsist

of two steps: arranging strategy parameters and then
modifying all of them by Eq. 4. Odup arranges in su h
a way that opies all of il (j ) for i0(l+1) (j ) and applies
D. Odel dis ards i0 (j ), opies il (j ) for i0(l 1) (j ). at

Table 1: Eleven test fun tions

Fun tions (n = 30)

f1 (x) =

n
X

(Range)

x2i

=1

100

 x  100)

0 (j ), the smaller value between max

the element
of im
Pm 1
and k=0 ik (j ) is inserted. Then, it mutates them
by D. Oinv swaps i0 (j ) with il (j ) and then mutates
by D. D is the same mutation as the Eq.(4) with a
lower bound. Se ondly, the omponent values x~i are
al ulated in the same manner as FEP, as follows:
x
~i (j )

f2 (x) =

n
X

jx j +

=1

f3 (x) =

n
Y

=1

jx j

10

 x  10)
i

i
n
X
X

=1 j =1

xj ) 2

100

 x  100)

100

 x  100)

f5 (x) =

n 1
X

=1

+1 x2i )2 + (xi

[100(xi

1)

30

 x  30)
i

f6 (x) =

n
X

=1

bx

+ 0:5 )

100

 x  100)
i

f7 (x) =

n
X

=1

fx2

10 os(2xi ) + 10

5:12

 x  5:12)
i

f8 (x) =

s
0:2

20 exp

exp

n
X

=1

n
X

x2i

!
(

=1!

32

 x  32)
i

os 2xi

+ 20 + e

1
f9 (x) = 4000

n
X

=1

x2i

n
Y

=1

os(

px

)+1

X

2
f
10sin (y1 ) +
(yi
n
i=1
[1 + 10sin2 (yi+1 ) + (yn
n

f10 (x) =

n
X

=1

1)

600

 x  600)
i

50

 x  50)

50

 x  50)

1)2 g

u(xi ; 10; 100; 4)

f11 (x) = 0:1fsin2 (3x1 ) +

n 1
X

(xi

1)

=1
[1 + sin22(3xi+1 )2
+(xn
1) [1 + sin (2xn )g
i

n
X

=1

u(xi ; 5; 100; 4)

where
yi = 1 + 14 (xi +(
1)
u(xi ; a; k; m) =

k(xi

k ( xi

a)m
a)

xi > a
a  xi  a
xi < a

xi (j ) + ~i0 (j )ij

(14)

where we alled i0 (j ) a tive strategy parameters, and

ina tive strategy parameters whi h are no ee t
of reating x~i . The proposed REP has the same omputational steps as CEP and FEP ex ept for the geneti operations des ribed above.
il (j )

f4 (x) = maxfjxi j; 1  xi  ng

4
4.1

COMPUTER SIMULATIONS

Test Fun tions and Conditions

Test fun tions are listed in Table 1. They are Sphere

Model (f1 ), S hwefel's Problem 2.22 (f2 ), S hwefel's
Problem 1.2 (f3 ), S hwefel's Problem 2.21 (f4 ), Generalized Rosenbro k's Fun tion (f5 ), Step Fun tion (f6 ),
Generalized Rastrigin's Fun tion (f7 ), A kley's Fun tion (f8 ), Generalized Griewank Fun tion (f9 ), Penalised Fun tion P8 (f10 ) and Penalised Fun tion P16
(f11 ). All of them are 30 dimensional problems where
the global minimums are 0. Fun tions f1 to f6 are unimodal fun tions, and the rest of them are multimodal
fun tions.
The experimental setup was done based on Yao and
Liu[8. All EP algorithms used the same sigma rst of
lognormal self-adaptation. The population size  was
set at 100. The tournament size for sele tion q is 10.
All simulations are independently repeated for 50 runs
using the same initial populations. The upper bound
of strategy parameters max is set at 3.0, ex ept for
1.0 for f7 . In REP, Odup , Odel and Oinv are applied
with the probabilities of 0.6, 0.3 and 0.1, respe tively.
The number of strategy parameters m for ea h variable is set at ve. The main purpose of our omputer
simulations is to investigate the ee t of lower bound
of strategy parameters on optimization performan e.
Thus, the parameters were not fully tuned.
4.2

Results

The results on unimodal fun tions of CEP, FEP and

REP are shown in Fig.1, Fig.2 and Fig.3, respe tively,
for ve lower bounds of 10 2 , 10 4 , 10 6 , 10 8 and
10 10 . The ee t of the lower bounds are observed
through the test fun tions. In Fig.1(a) and Fig.2(a),
after approximately 200 generations the smaller lower
bounds make onvergen e rates of CEP and FEP
worse. On the ontrary, in ase of REP shown in
Fig.3(a), better performan e are obtained with the
same onvergen e rate when the smaller lower bound
is adopted. In Fig.1(b), Fig.2(b) and Fig.3(b) the results on f2 show the same tenden y as on f1 . After
approximately 300 generations, smaller lower bounds

Table 2: The numbers of trials that EP does not get trapped in lo al optima.
exp()

2
4
6
8
10

f7
FEP
0
24
0
0
0

CEP
0
0
0
0
0

REP
0
50
50
50
50

CEP
0
0
0
0
0

f8
FEP
0
50
34
2
0

REP
0
50
50
50
50

CEP
0
12
12
12
12

bring worse onvergen e rates of CEP and FEP. On

the ontrary, in ase of REP, better performan e are
obtained with the same onvergen e rate when the
smaller lower bounds are adopted. In Fig.1( )(d)(e)
and Fig.2( )(d)(e), on f3 after approximately 400 generations, on f4 after approximately 500 generations
and on f5 after approximately 200 generations, respe tively, the smaller lower bounds are, the worse onvergen e rates of CEP and FEP are. On the ontrary,
in ase of REP shown in Fig.3( )(d)(e), better performan e are obtained with nearly the same onvergen e
rate when the smaller lower bounds are adopted. In
Fig.1(f), Fig.2(f) and Fig.3(f) the results of f6 whi h
is dis ontinuous fun tion show that REP an nd the
global optimum with ea h lower bound, but CEP and
FEP an not nd the global optimum ex ept for FEP
with the lower bound 10 2 .
The results on multimodal fun tions of CEP, FEP
and REP are shown in Fig. 4, Fig.5 and Fig.6, respe tively, for the same set as the experiment on the unimodal fun tions. In Fig.6(a), (b), (d) and (e) results of
REP, similar to those in the unimodal fun tion's ase,
f1 and f2 , are shown. In these ase, the better results
are shown by using of smaller lower bounds. As shown
in Fig.5, FEP showed stagnant onvergen e, ex ept for
only  = 10 4 on f7 , f8 and f11 . As shown in Fig.4,
CEP showed stagnant onvergen e, ex ept for f9 . As
shown in Table 2, REP on f7 , f8 , f10 and f11 does not
get trapped in lo al optima at all, provided that the
lower bound are smaller than 10 4 . Con erning to f9 ,
REP has higher frequen y of not getting trapped in
lo al optima than CEP and FEP.
5

CONCLUSIONS

This paper proposed an extended EP with non- oding

segments, alled Robust Evolutionary Programming
(REP). The motivation was that the ee t of geneti
drift an be added as another sour e of hanging strategy parameters, be ause they are not su iently ontrolled by natural sele tion. Computer simulations
were ondu ted on eleven test fun tions to evaluate
the optimization performan e. The results suggested
that the lower bound is a riti al parameter for both
of CEP and FEP, while REP is almost free from this
ondition.
The future work will be the analysis of the evolu-

f9
FEP
0
5
3
3
5

REP
0
21
17
19
18

CEP
0
22
23
18
8

f10
FEP
0
48
42
31
10

REP
0
50
50
50
50

CEP
0
14
15
15
14

f11
FEP
0
50
47
41
26

REP
0
50
50
50
50

tionary dynami s by s aling up and down the size of

sear h spa e and the evaluation of noise ee t to the
optimization performan e, in onsideration of solving
real-world design problems, su h as re urrent neural
networks and industrial me hani al design problems.
ACKNOWLEDGEMENTS

Thanks to nan ial support through the \Methodology

of Emergent Synthesis" proje t(96P00702) by JSPS
(the Japan So iety for the Promotion of S ien e).

REFERENCES
[1 T. Ba k and H.-P. S hwefel (1993), \ An Overview of
Evolutionary Algorithms for Parameter Optimization",
Evolutionary Computation, Vol. 1, No. 1, pp. 1-23.
[2 K. Chellapilla (1998), \Combining Mutation Operators
in Evolutionary Programming", IEEE Transa tions on
Evolutionary Computation, Vol. 2, No. 3, pp.91-96.
[3 D. B. Fogel (1995), Evolutionary Computation Toward
a New Philosophy of Ma hine Intelligen e , IEEE Press.
[4 M. Kimura (1983), The Neutral Theory of Mole ular
Evolution, Cambridge University Press.
[5 K.-H. Liang, X. Yao, Y. Liu, C. Newton and D. Homan (1998), \ An Experimental Investigation of Selfadaptation in Evolutionary Programming", Pro . of
7th Annual Conferen e on Evolutionary Programming,
pp.291-300, Springer-Verlag.
[6 K.-H. Liang, X. Yao and C. Newton (1998), \Dynami
Control of Adaptive Parameters in Evolutionary Programming", Pro . of the Se ond Asia-Pa i Conferen e on Simulated Evolution and Learning, SpringerVerlag.
[7 N. Saravanan and D. B. Fogel (1997), \Multi-operator
Evolutionary Programming: A Preliminary Study on
Fun tion Optimization", Pro . of 6th Annual Conferen e on Evolutionary Programming, pp.215-221,
Springer-Verlag.
[8 X. Yao and Y. Liu (1996), \Fast Evolutionary Programming", Pro . of the Fifth Annual Conferen e on Evolutionary Programming, pp 451-460, MIT Press.
[9 X. Yao and Y. Liu (1998), \ S aling up Evolutionary Programming Algorithms", Pro . of the 7th Annual
Conferen e on Evolutionary Programming, pp 103-112,
Springer-Verlag.

100000

100000
2
4
6
8
10

1e-10

1e-05
1e-10

1e-15
1000

2000
3000
Generation

4000

5000

1e-15
0

(a) CEP on f1
2
4
6
8
10

1e-06

1e-09

1e-09
1000

2000

3000

4000

5000

2000

3000

4000

5000

2
4
6
8
10

1000

5000

2000
3000
Generation

4000

2
4
6
8
10

1
0.1

2000
3000
Generation

4000

5000

1000

(d) CEP on f4

2000
3000
Generation

4000

5000

10000

Value

Value

10000
1000

1000
100

10

10

10

1
4000

5000

1000

2000
3000
Generation

4000

5000

100

0.01
600
900
Generation

1200

1500

4000

5000

2
4
6
8
10

100

0.01
300

2000
3000
Generation

10000

Value

Value

1000

(e) REP on f5
2
4
6
8
10

10000

100

(e) FEP on f5
2
4
6
8
10

10000

1
0

(e) CEP on f5

5000

2
4
6
8
10

100000

100

2000
3000
Generation

4000

(d) REP on f4

100

1000

2000
3000
Generation

1e+06
2
4
6
8
10

100000

1000

1000

(d) FEP on f4
1e+06
2
4
6
8
10

10000

0.1

0.001
0

1e+06
100000

0.01

0.001
1000

5000

2
4
6
8
10

10

0.01

0.001

4000

( ) REP on f3

Value

0.1

2000
3000
Generation

100

10

0.01

1000

( ) FEP on f3

Value

2
4
6
8
10

1000

5000

100
2
4
6
8
10

5000

0.001
1000

( ) CEP on f3
10

4000

100

3000

1e+06

Value

Value
4000

2000

(b) REP on f2

0.001
2000
3000
Generation

1000

Generation

1000

1e-09
1000

1e+06

0.001

5000

0.001

(b) FEP on f2

4000

2
4
6
8
10

1000

Generation

1000

2000
3000
Generation

1e-06

2
4
6
8
10

1000

(a) REP on f1

0.001

1e-06

1e+06

Value

Value

(b) CEP on f2

Value

5000

2
4
6
8
10

Generation

Value

4000

1e+06

1000

0.001

Value

2000
3000
Generation

1e+06

Value

Value

1000

(a) FEP on f1

1e+06
1000

1e-05
1e-10

1e-15
0

2
4
6
8
10

Value

1e-05

Value

Value

100000
2
4
6
8
10

0.01
0

300

600
900
Generation

1200

1500

300

600
900
Generation

1200

1500

(f) CEP on f6

(f) FEP on f6

(f) REP on f6

Figure 1: The averaged best results of CEP on unimodal fun tions when the lower bounds are
10 2 , 10 4 , 10 6 , 10 8 and 10 10 .

Figure 2: The averaged best results of FEP on unimodal fun tions when the lower bounds are
10 2 , 10 4 , 10 6 , 10 8 and 10 10 .

Figure 3: The averaged best results of REP on unimodal fun tions when the lower bounds are
10 2 , 10 4 , 10 6 , 10 8 and 10 10 .

2
4
6
8
10

1e-10

1e-05

1e-10

1e-15
1000

2000

3000

4000

5000

1e-15
0

1000

Generation

(a) CEP on f7

5000

3000

4000

0.001

5000

1000

2000

3000

4000

5000

(b) FEP on f8
2
4
6
8
10

Value

Value

Value

2000
3000
Generation

4000

5000

0.1

0.001
0

1000

( ) CEP on f9

4000

5000

1e-05

1e-10

1e-15

1e-15

1e-15

1e-20

1e-20
4000

5000

1e-20
0

(d) CEP on f10

2000
3000
Generation

4000

5000

1e-05

1e-05
1e-10

1e-10

1e-15

1e-15

2000
3000
Generation

4000

5000

1000

2000
3000
Generation

5000

1e-05

1e-15
1000

4000

4000

2
4
6
8
10

1e-10

2000
3000
Generation

100000

Value

1000

(d) REP on f10

2
4
6
8
10

100000

Value

1000

(d) FEP on f10

2
4
6
8
10

100000

5000

1e-05

1e-10

2000
3000
Generation

4000

2
4
6
8
10

1e-10

1000

2000
3000
Generation

100000

Value

1e-05

1000

( ) REP on f9

2
4
6
8
10

100000

Value

2000
3000
Generation

( ) FEP on f9

2
4
6
8
10

100000

5000

2
4
6
8
10

10

0.001
1000

4000

(b) REP on f8

0.1

0.001

3000

1000

10

0.1

2000

Generation

1000

1000

Generation

(b) CEP on f8
10

0.001

1e-09
0

2
4
6
8
10

5000

1e-06

Generation

1000

4000

2
4
6
8
10

1e-09
2000

3000

(a) REP on f7

1e-06

1000

2000

1000

Value

0.001

1000

Generation

2
4
6
8
10

Value

Value

4000

(a) FEP on f7

1e-09

Value

3000

1000
2
4
6
8
10

1e-06

Value

2000

Generation

1000

1e-05

1e-10

1e-15
0

2
4
6
8
10

Value

1e-05

2
4
6
8
10

Value

Value

5000

1000

2000
3000
Generation

4000

5000

(e) CEP on f11

(e) FEP on f11

(e) REP on f11

Figure 4: The averaged best results of CEP on unimodal fun tions when the lower bounds are
10 2 , 10 4 , 10 6 , 10 8 and 10 10 .

Figure 5: The averaged best results of FEP on unimodal fun tions when the lower bounds are
10 2 , 10 4 , 10 6 , 10 8 and 10 10 .

Figure 6: The averaged best results of REP on unimodal fun tions when the lower bounds are
10 2 , 10 4 , 10 6 , 10 8 and 10 10 .