2011 International Symposium on Computer Science and Society
Improving BP Neural Network for the Recognition of Face Direction
Ying He Baohua Jin
Henan Business College Zhengzhou University of
Zhengzhou, China Light Industry
hying2050@yahoo.com.cn Zhengzhou, China
jinbh@zzuli.edu.cn
Abstract - the recognition of face direction is an important
part of the artificial intelligence. In recent years, BP network
has been used for pattern recognition. However, in practical
application, BP has some disadvantage. The widely used BP
algorithm has slow convergent speed and learning efficiency,
and it is easy to get into local minimum. Selection of the
initial value of the BP network can also affect convergent
speed. This paper presents an improving BP network to
accelerate convergence with genetic-simulated annealing
algorithm. So, we optimized the initial value of the network
through adding annealing idea into genetic
algorithm(Genetic-stimulated annealing algorithm, GSA) to
identify face direction. Using this improving BP neural
network for the recognition of face direction, the results
presents that our method has the highest precision and
reaches relatively good effects compared with traditionally
BP neural network.. Therefore, this method optimized with
GSA poses better recognition ability, and achieves ideal
effect for the face direction.
Keywords - face direction; BP network; the initial value of
the network; Genetic-stimulated annealing algorithm
I. INTRODUCTION
As one of the most hopeful applications of image
analysis and understanding, face recognition has recently
received significant attention. BP network method has
been applied to recognize face direction. It has very strong
computing capability and is suitable for being applied to
complicated non-linear environment. But BP neural
network has itself drawbacks slow convergent speed and
easily getting into local minimum.
To improve BP neural network method, many
scholars have presented many methods to accelerate
convergent speed and search global optimum to overcome
the shortcoming of BP neural network. T.P. Voglet.al. first
introduced several methods for accelerating the
convergence of back-propagation. It included batching,
momentum (MOBP), variable learning rate (VLBP) [1]. In
the same time, because of the development of heuristic
techniques and numerical optimization techniques, these
techniques were gradually applied to neural network,
especially genetic algorithm (GA) and stimulating
annealing algorithm (SA). GA was invented by John
Holland [2]. SA was a probabilistic method proposed by
Kirkpatrick, Gelett, Vecchi and Cerny .
GA is a robust adaptive optimization based on
biological principles. And SA is a kind of random search
technique based on the probability. GA is poor in the local.
In contrast, SA has a strong local search capability and
makes the search process to avoid falling into local
optimal solution. For these reasons, we adopt the method
by combining GA with SA to complement each other and
optimize the initial weights and biases value of BP neural
network in this paper.
The remainder of this paper is organized as follows.
978-0-7695-4443-4/11 $26.00 2011 IEEE 79
DOI 10.1109/ISCCS.2011.29
Qiongshuai Lv Shaoyu Yang
Zhengzhou University Zhengzhou University
Zhengzhou, China Zhengzhou, China
qiongshuailv@163.com ysymagnet@gmail.com
Section briefly introduces the related literature review,
while Section describes GSAs idea and analyzes the
internal process. Section discusses the experiment and
presents the primary results. Finally, Section provides
the summary and conclusions.
II. RELATED LITERATURE REVIEW
A.BP Neural Network
The basic idea of BP is that the learning process is
composed of two processes including positive diffusion of
signal and inverted diffusion of error. Generally, BP
neural network consists of three layers: input layer, hidden
layer and output layer. Topology is illustrated in Fig.1.Pi is
input vector, i {1,..., n} , Wij is the weight between
input layer and hidden layer. Wjk is the weight between
hidden layer and output layer. ai is output vector,
i {1,..., m} .
Figure 1. n-r-m neural network structure
Recognition on face direction by BP network is
mainly determined network structure. It has been proved
theoretically that the three-layers feed forward network
can be trained to approximate arbitrary function well.
Determine the number of nodes in each layer, we take the
following rules [4]:
z The number of inputs to the network and the number
of outputs from the network are defined by external
problem specifications.
z The external problem does not tell us directly the
number of nodes required in the hidden layers. In
fact, there are few problems for which one can
predict the optimal number of nodes needed in a
hidden layer. This problem is an active area of
research.
As for the step 2 people often use the following three
formulas [5] to determine the number of nodes in the hidden
layer.
n
C
t 0
i
nt !k
(1)
Here k is the number of sample. ni is the number of nodes
in the hidden layer. n is the number of nodes in the input
layer. If i ni , Cni
i
0.
 nt nm a
(2)
Here m is the number of nodes in the output. n is the
number of nodes in the input layer. a is the constant
between 1 and 10.
nt log 2 n
(3)
Here n is the number of node in the input layer.
As we all known, BP network training is the continuous
process of learning. The core content of the learning
process is to adjust weight matrix according to the mean
square error between the real output and anticipant output.
B.Genetic Algorithm
Genetic algorithm (GA) is a kind of search technique
used in computing to find exact or approximate solutions
to optimize and search problems. The genetic algorithm is
categorized as global search heuristics. A typical genetic
algorithm requires:
z A genetic representation of the solution
domain.
z A fitness function to evaluate the solution
domain.
The fitness function is defined over the genetic
representation and measures the quality of the represented
solution. Once we have the genetic representation and the
fitness function defined, GA proceeds to initialize a
population of solutions randomly, and then improve it
through repetitive application of selection, crossover and
mutation operators. The training process forward neural
network is in fact a sort of based on previous three
operators to look for optimal link weight value and bias
value. Each group of weight value and bias value
represents the GA as the individual chromosome. So,
simple GA pseudo code is shown as follows [6]:
z Choose the initial population of individuals.
z Evaluate the fitness of each individual in that
population.
z Repeat on this generation until termination:
(time limit, sufficient fitness achieved, etc.).
1. Select the best-fit individuals for reproduction.
2. Breed new individuals through crossover and
mutation operations to give birth to offspring.
3. Evaluate the individual fitness of new
individuals.
4. Replace least-fit population with new
individuals.
C. Stimulated Annealing Algorithm
The inspiration of simulated annealing (SA) [7] comes
from annealing in metallurgy. a technique involving
heating and controlled cooling of a material increases the
size of its crystals and reduces their defects. The heat
causes the atoms to become unstuck from their initial
positions (a local minimum of the internal energy) and
wander randomly through states of higher energy, while
the slow cooling gives them more chances of finding
configurations with lower internal energy than the initial
one.
By analogy with this physical process, each step of
the SA algorithm replaces the current solution by a
random "nearby" solution, chosen with a probability that
depends on the difference between the corresponding
80
function values and also on a global parameter T (called
the temperature), which is gradually decreased during the
process. The dependency is such that the current solution
changes almost randomly when T is large, but increasingly
"downhill" as T goes to zero. The allowance for "up
hilling" moves to save the method from becoming stuck at
local optima - which are the bane of greedier methods.
As described above, the following pseudo code
implements the simulated annealing algorithm:
z Randomly generate an initial point, and
calculate the objective function value.
z Set the initial temperature and counter T.
z Do a random change in the current point,
calculate the new objective function value and
the increment .
z If 0, accept the current new point as the
current point. If  0, accept the current new
point in probability P as the current point.
z If T initial value, T = T + 1, jump to step
3.
z If the temperature is not satisfactory, jump to
step 2.esle return the current optimal point.
III. GENETIC-STIMULATED ANNEALING ALGORITHM
Through theoretical analysis of the previous, GA has
an enough strong global search ability, but poor in the
local. In contrast, SA has a strong local search capability
which can make the search process avoid falling into local
optimal solution. However, SA knows a little for the
whole search space. For these reasons, we adopt the
method by adding annealing idea into genetic algorithm to
complement each other and optimize the initial weights
and biases of BP neural network.
GSA is similar to GA in the overall operation. It also
needs three basic operators including selection, crossover
and mutation to produce a new group of individuals. Then,
implementation of the SA processes in each individual
independently. Its results as the next generation of group
of individuals. This process is repeatedly carried out until
satisfy the end condition. Fig. 2 shows the basic flowchart
of GSA on the BP neural network.
In the right picture, it includes two parts: one is the
BP neural network and the other is the GSA algorithm.
The part of BP neural network use optimized weights and
biases with GSA algorithm. Usually, the initial weights
and biases of BP neural network is selected randomly. we
use the processed value by GSA algorithm. And the other
part of GSA algorithm mainly including the following
three parts:
A..Fitness Function
GSA is not basically used for external information in
the search process, but only on the basis of the fitness
function. It takes advantage of the fitness function of each
individual value to search in the population. Fitness
function affects the convergence speed directly. Generally
speaking, fitness function is formed by the objective
function, we regard error between the anticipate output
and real output as individual fitness.
B.Three operation
In the selection operation, we choose individual
according to fitness value. The greater fitness of
individuals is, the more likely to be selected. Our
experimental adopts the way of roulette wheel selection.
In the crossover operation, we randomly select two
individual chromosomes to cross. Our experiment take the
floating encoding in the process of individual chromosome
coding. So, we take advantage of float cross rules. In the
mutation, we randomly choose the i-th individuals of the
j-th genes to mutate. These three operators are the main
part of genetic algorithm and can guarantee to GSA have
enough strong global search ability.
C. SA Operator
In the SA operator, we use Metropolis receive rule [8]
and parameter evolution of anneal process to run program.
Based on this point, SA operator just makes up the
deficiency of GA in the local search capability. Metropolis
receive rule is shown as follows:
^
| E1 | t | E 2 |
1 to determine the optimal number of nodes in the hidden
P exp(| E1 |  | E 2 |) / T | E |  | E |
1 2
(4)
Here P is the Metropolis receiving probability. E1 is the
old energy.E2 is the new energy. When the E1 is less than
E2, we accept this change. Or we receive this change with
probability exp(|E1|-|E2|).
T is the annealing value.
T T0 / log(T  1) (5)
Here T0 is the initial temperature.
Metropolis receive rule can make the current solution
break out of local minimum. In a word, SA operator has a
strong local search capability which can make the search
process avoid falling into local optimal solution.
These are the core of the idea of GSA algorithm. GA
has an enough strong global search ability, but poor in the
local. In contrast, SA has strong local search capabilities
and makes the search process to avoid falling into local
optimal solution. However, SA knows a little for the
whole search space. In this case, we adopt the method by
adding annealing idea into genetic algorithm to
complement each other and optimize the initial weights
and biases of BP neural network.
Figure 2. GSA flowchart based on BP network
IV. EXPERIMENT AND SIMUTLATION
A.Experiment
81
1).Pretreatment on Feature Data of face image
In this paper, our experimental data set is from:
http://archive.ics.uci.edu/ml/index.html. In observation,
we noticed that the location of the eyes have significant
differences when face direction is not the same, which has
five directions, including the left , the left front, the front,
the right front and the right. We use three binary to
express facial direction. We divided pictures into six rows
and eight columns, and then marginalized pictures. we
only extract the rows and columns where the eye position.
We counted the number which the pixel of eyes position
is one. To ensure the randomness of the training data, we
chose 30 as the training set and 20 as the testing set.
2).Determination of BP Network Structure
The idea of this experiment is to design a three-layer
BP network. It includes eight nodes in the input layer and
three nodes in the output layer. It is an unsolved problem
layer. If the nodes are not enough, the convergence speed
of the whole network will slow down. On the contrary, if
there are excessive nodes, he BP network will be more
calculation work on iterative study and error may not be
up to grade. we take the formula (2) in section to
determine. So, the BP neural network structure is
eight-ten-three.
3).Weight value Encoding Mechanism
According to our established network structure, eight
nodes in the input layer, ten nodes in the hidden layer and
three nodes in the output layer, and the main part of the
GSA is GS. So the length of individual chromosome is
123 in the GSA.
We take the floating-point encoding in the process of
individual chromosome coding. Floating-point encoding
can help improve operational efficiency and computational
complexity. Each individual is a floating-point string. It
consists of four parts, which are the input layer and hidden
layer connection weights, hidden layer biases, the hidden
layer and output layer connection weights and the output
layer biases. Individual chromosomes contain all of the
neural network weights and biases.
4). Experimental environment and Set Parameter
Use Matlab toolbox to realize BP neural network in
PC Pentium4/CPU 2.8GHz/RAM512MB.With GSA
algorithm optimal BP neural network. In the GSA
algorithm, we set the population size of 10.The number of
iteration is 20. Crossover probability is 0.4. Mutation
probability is 0.2. The initial temperature T0 is 1000.
B.Simulation
We compared the error between GSA algorithm
optimal BP neural network and original BP network.Fig.3
shows the result.
Utilize the GSA algorithm to optimize neural
network, and optimize the neural network weights and
biases. From the figure 3, we can clearly see the error
distribution situation of simulation result for the testing
sample set after training network. Although it is closely
that we use two method, the error of recognition with GSA
algorithm optimal BP neural network is obviously smaller
than the original BP network. This way has the highest
precision and reaches relatively good results.
 V. CONCLUTIONS
This paper concentrates on the random initial weight
value and bias value of traditional BP neural network,
such as easy to trap local minimal and has slowly leaning
convergent velocity. we use a method based on GSA
algorithm to optimize its initial values, aimed at improving
the velocity of learning convergence. This method is
practical and general applicable. When making
recognition of face direction, firstly, it should locate the
position of eyes in the picture to reduce the influence of
other factors and improve the recognition precision. All of
these have proved that comparison with the traditional BP
network, the GSA optimization of BP network could get a
[3] Feng Li, Cheng Liu, Application Study of BP Neural Network on
Stock Market Prediction, Ninth International Conference on Hybird
Intelligent System, Vol. 3, pp. 174-178, August, 2009
[4] Martin T. Hagan, Howad B. Dcmuth, Mark Beale, Neural Network
Design, China Machine Press, Beijing, 2002
[5] Feisi ICS-UNIDO, Neural Networks and Matlab 7 Application,
Publishing House of Electronics Industry, Beijing, 2005
[6] Tom M. Mitchell, Machine Learning, China Machine Press, Beijing,
2003
[7] Kirkpatrick. S, Gelatt. C. D, Vecchi. M. P. Optimization by
simulated annealing, Science, pp.671-680,1983
[8] Tian Qi Chuan,Pan Quan,Wang Feng,Zhang Hong Cai, Research
on Learning Algorithm of BP Neural Network Based on the
Metropolis Criterion,Control Theory and Applications,
Vol.22.pp.15-17,2003
Recognization Error

much more accuracy result. GSA - Error

0.9
BP - Error
Although the improved BP neural network has
0.8
exceeded the traditional algorithm in convergence speed
0.7
and recognition precision, this method needs a large
0.6
amount of calculation. Pattern recognition problems of the
0.5
real world, such as dynamic image recognition, do not fit

Error
well with this narrowly-defined model. They tend to span 0.4

broad activities and require consideration of multiple 0.3

aspects. 0.2

REFERENCES 0.1
[1] T. P. Vogl, J. K. Mangis, A. K. Rigler, W. T. Zink, D. L. Alkon, 0
Accelerating the Convergence of the Back-Propagation
-0.1
Method,Biological Cybernetics, Vol. 59, pp. 257-263, September, 2 4 6 8 10 12 14 16 18 20
1988 Test Sample
[2] David E. Goldberg, John H. Holland, Genetic Algorithms and Figure 3 Recognization Error
Machine Learning, Machine Learning, Vol. 3, pp. 95-96, , October,
1988

82