http://www.paper.edu.
cn
BP Neural Network principle and MATLAB Simulation
Xiong Xin
School of Information Engineering
Wuhan University of Technology
Wuhan, P. R. China 430070
sense1559@163.com
Abstract
This paper introduces the prevalent BP algorithm in neural network, and discusses the goodness
problem and training process of BP neural network, as well as using MATLAB software to simulate
the numbers on the basis of it. At last, several improved BP training algorithms have been compared
in the paper.
Keywords: BP neural network; number recognition; MATLAB
1 Introduction
The development of neural network is
rapid since the first neural network model
[1]
MP model came up in 1943 . Hopfield
neural network proposed in 1982 and opposite
phase broadcast algorithm proposed by
Rumelhart in 1985 make the neural network of
Hopfield model and multilayer feedforward
model to be the prevalent neural network
model. They are effective in many
applications of fields such as speech
recognition, mode recognition, image
processing and industry controlling.
Neural network is an theory which is
imitative of the biological processing model to
get the function of information intelligent
processing. It treats with the pattern
information which is hard to be expressed in
certain language by the method from bottom
to top and parallel distribution way formed by
self-study, self-organization and non-linear
dynamics. Neural network is a parallel and
distributed information processing network
architecture. It is generally composed by
massive neurons, each of which has only one
output that can connect many other neurons.
The reciprocity between neurons is embodied
by their connected weighs. The output of
neurons is its input function. The types of
functions in common use have linear function,
Sigmoid function and value function.
There are two phases of positive
transmitting processing and error reverse
-1-
Nie Mingxin
School of Information Engineering
Wuhan University of Technology
Wuhan, P. R. China 430070
niemx@sohu.com
transmitting processing in the study
[2]
processing of BP neural network . The
signal inputted from outside spreads to the
output layer and gives the result through
processing layer for layer of neurons in input
layer and hidden layer. If the expected output
cant be obtained in output layer, it shifts to
the conversed spreading processing and the
true value and the error outputted by network
will return along the coupled access formerly.
The error is reduced by modifying contacted
weight value of neurons in every layer and
then it shifts to the positive spreading
processing and revolves iteration until the
error is smaller the given value. Take a three
layer network for example, the network is
composed of N input neurons, K hidden
neurons and M output neurons(as showed in
fig.1). O2pm and O1pk are the output value of
output layer and hidden layer respectively.
w2km and w1nk are the connected weight
value from the hidden layer to the output layer
and from the input layer to the hidden layer
respectively. Suppose the input studying
sample is Xpn, so its corresponding expected
output value is tpm.
Fig.1 BP neural network configuration

2 BP Neural Network
2.1 the Discussion about the Advantages
and Disadvantages of BP Neural Network
BP neural network is a kind of neural
network forms which has most applications
[3]
currently , but it isnt very perfect. In order
to understand how to apply the neural network
to resolve problems, we carry on the
discussion about its advantages and
disadvantages here.
The advantages of BP neural network:
Network realizes the mapped function from
input to output and mathematical theory has
proved that it has the function to achieve
any complex non-linear mapping;
Network can extract the logical solution
rules automatically through studying the
examples with correct results. It has the
ability of self-study;
Network has the definite abilities of
promotion and generalization.
The disadvantages of BP neural network:
The study speed of BP algorithm is very
slow. The main causations of it are:
a. Because BP algorithm is grads declining
method essentially and the aim function
optimized by it is very complex, the
sawtooth-shaped phenomenon is bound to
appear which makes the BP algorithm
inefficiency;
b. The torpid phenomenon exists. Because the
optimized aim function is very complex, it
can appear some flat areas in the case of the
output of neurons approach 0 or 1. In these
areas, the error of weight value changes
very little, which can hardly make the
training processing break down;
c. In order to execute the BP algorithm in the
network, we cant use the traditional
one-dimensional search method to solve the
interative step length every time. We should
put the network the updated rules of step
length in advance. The former algorithm will
make the algorithm inefficiency.
the network training is much more likely to
fail, the reasons as below:
-2-
http://www.paper.edu.cn
a. From the perspective of mathematics, BP
algorithm as a kind of local searching
optimized method, it is used to solve the
overall extremum of complex non-linear
function, so the algorithm is likely to be gotten
into the local extremum and make the training
fail;
b. The approaching and promoting abilities are
closely linked with the representative of
studying sample. It is a hard problem to
choose the training collection composed of
the typical sampling examples.
The contradiction between the scale of
examples and network is hard to solve, which
refers to the relationship of possibility and
feasibility of network capacity, viz. the
problem of studying complexity;
The choice of network configuration has
still no a uniform and integrate academic
guidance and it can be selected by experience.
Therefore some people call the structure
choice of neural network is a kind of art. The
network structure infects the approaching
ability and promoting character directly. So
how to choose an appropriate network
structure is an important problem;
New samples can infect the network which
studies successfully and the number that
describes the character of every input
sample should be equal;
There is contradiction between the
predictive ability of network(also called
generalization ability or promoting ability)
and training ability(also called approaching
ability or study ability). Usually when the
training ability is poor, the predictive ability
will be poor and in a certain extent, with the
improvement of training ability the predictive
ability is also improved. However, this trend
has a limit. When achieving this limit, with
the improvement of the training ability the
predictive ability will be decline on the
contrary, which is also called the over fitting
phenomenon. And now the network studies

too much detail of samples and cant reflect
the embedded laws of samples.
2.2 BP Network algorithm
The training process of BP network is as
[4]
below .
(1) Initialization. Endow every connected
weight value and threshold value with a lesser
random value.
(2) Input the corresponding neurons in input
layer with an component of a eigenvector Xpk
= (Xpk1 ,Xpk2 Xpk3 ,,Xpkn) .
(3) Use the eigenvector of input samples to
calculate the corresponding output value
O1pk=f(Xpkn) of neurons in hidden layer.
(4) Use each unit output O1pk in hidden layer
to calculate the input value in each output
layer and then further calculate the
corresponding output O2pm ,O2pm
=f(O1pk) of each unit in output layer.
(5) Calculate the generalized error of each unit
in output layer via the teach signals.
(6) Use the connected weight value W2km
between middle layer and output layer, the
generalized error i in each unit of output
layer and the output O1pk of each unit in
middle layer to calculate the generalized error
of each unit in middle layer.
(7) Use the generalized error j of each unit in
output layer and the output O1pk of each unit
in middle layer to modify the weight value
w2km between output layer and middle layer
and the threshold value Yj of each unit in
output layer.
(8) Use the generalized error ei of each unit in
middle layer and the input Xpkn of each unit
in input layer to modify the weight value
w1nk between input layer and middle layer
and the threshold value i of each unit in
hidden layer.
(9) Select the next sample in order and return
2 until the samples in training collection are
all studied over.
(10) Return 2 over again until the error
function is lower than the predetermined value,
viz. the times of network constringency or
-3-
http://www.paper.edu.cn
study is greater than the predetermined value.
3 the Applications of BP Network in
digital recognition
Firstly, the numbers 0 9 should be
carried on the digital processing to constitute
the input samples. Considering that using the
0-1 image of 55 matrix will represent every
number clearly and then it designs and trains
an neural network which can recognize the ten
numbers 0~9 once again. When giving the
trained network with an input that can
represent some number, network can represent
this number correctly through the 8421BCD
code in output terminal. This network can
remember all the ten numbers by study and
training. The neural network training should
be supervised to train ten groups of arrays
which can represent numbers 0~9 and show
the corresponding four two-scale numbers of
1~10 in output terminal.
3.1 the Design of network structure
From the analysis above, we can get that
the neural network need to have N=55 input
neurons and M=4 output neurons. The two
layer logsig/logsig network adopting the
logarithm Sigmoid type active function in the
scope of (0,1) is quite effective to the 0,1
Boolean values. So the network may adopt
the N-K-M structure, where the hidden layer
chooses one layer, N is the number of
neurons in input layer, K is the number of
neurons in hidden layer and M is the number
of neurons in output layer. The number 0
and 9 can represent respectively by 0-1
chart as:
le0=[1 1 1 1 1 le1=[0 0 1 0 0
1 0 0 0 1 0 0 1 0 0
1 0 0 0 1 0 0 1 0 0
1 0 0 0 1 0 0 1 0 0
1 1 1 1 1] 0 0 1 0 0]
le2=[1 1 1 1 1 le3=[1 1 1 1 1
0 0 0 0 1 0 0 0 0 1
1 1 1 1 1 1 1 1 1 1
1 0 0 0 0 0 0 0 0 1
1 1 1 1 1] 1 1 1 1 1]
le4=[1 0 0 0 1 le5=[1 1 1 1 1
1 0 0 0 1 1 0 0 0 0

1 1 1 1 1 1 1 1 1 1
0 0 0 0 1 0 0 0 0 1
0 0 0 0 1] 1 1 1 1 1]
le6=[1 1 1 1 1 le7=[1 1 1 1 1
1 0 0 0 0 0 0 0 1 0
1 1 1 1 1 0 0 1 0 0
1 0 0 0 1 0 1 0 0 0
1 1 1 1 1] 1 0 0 0 0]
le8=[1 1 1 1 1 le9=[1 1 1 1 1
1 0 0 0 1 1 0 0 0 1
1 1 1 1 1 1 1 1 1 1
1 0 0 0 1 0 0 0 0 1
1 1 1 1 1] 1 1 1 1 1]
3.2 Network MATLAB Simulation
To the applied BP network above, we can
use the functions in MATLAB neural network
[5]
toolbox to simulate . The neural network
toolbox is one of toolboxes under the
MATLAB environment. It constructs the
excitation functions of typical neural network
such as the S type, linear type, competitive
type and saturated linear type functions based
on the artificial neural networks theory and
using the MATLAB language. These
functions can make the designer change the
selected network output calculation to the
transfer of excitation functions.
Hidden layer gets 10 neurons relied on
experience. To the network structure which is
25-10-4 structure, input layer has 25 neurons
and each number can be represented with 0-1
chart of 55 matrix whose elements can
constitute a numeric column matrix. The 10
numbers are represented separately by a 25
10 input matrix which is composed of 10
numbers column matrix. And then send the 10
numbers to the variable named P:
P= [le0, le1, le2, le3, le4, le5, le6, le7, le8,
le9]
Output layer has 4 neurons, because the
objective vector expects that when each
number is inputted, the four binary elements
of 1~10 can be corresponded in the output
terminal correctly.
The design parameters of network are: the
largest training times 5000-2000(according
to the training function); no noise recognized
-4-
http://www.paper.edu.cn
training precision 0.1; the rate of study 0.01;
momentum constant 0.95; the biggest error
ratio 1.05.
The program flow as below:
le0=[1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1
1 1 1]';
le1=[0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0
1 0 0]';
le2=[1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 1 1
1 1 1]';
le3=[1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 1
1 1 1]';
le4=[1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0
0 0 1]';
le5=[1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 1
1 1 1]';
le6=[1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 0 0 0 1 1 1
1 1 1]';
le7=[1 1 1 1 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0
0 0 0]';
le8=[1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 1
1 1 1]';
le9=[1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 1
1 1 1]';
P=[le0,le1,le2,le3,le4,le5,le6,le7,le8,le9];
T=[0 0 0 0;0 0 0 1;0 0 1 0;0 0 1 1;0 1 0 0;0 1 0
1;0 1 1 0;0 1 1 1;1 0 0 0;1 0 0 1]';
%adopting feedforward BP network algorithm
%and the BP algorithm training function
%Levenberg_Marquardt
net=newff(minmax(P),[10,4],{'logsig','logsig'}
,'trainlm');
%the biggest training times is 5000 here
net.trainParam.epochs=5000;
net.trainParam.goal=0.1;
net.trainParam.lr=0.01;
net.trainParam.mm=0.95;
net.trainParam.er=1.05
net=train(net,P,T);
The training curve that come out finally is
shown in fig.2, in which we can that the
network converges to the required precision at
a great speed. In the condition that objective
convergent precision and other training
 http://www.paper.edu.cn

parameter dont change, compare the Levenberg_

simulations separately of the other several BP trainlm Marquardt 5000/ 0.0994116
algorithm training functions, the result shown BP algorithm 7
in table 1. In this table we can conclude that in The adjust of Levenberg Marquardt
[6]
the condition of not affecting the precision, algorithm weight value is :
adopting the L-M optimized algorithm is the 1
most fast.
Where J is a Jacobian matrix which is
composed of the differential coefficient by
error to weight value, E is the error vector, m
is a scalar quantity which depends on the
width of m. This method is ranging between
the two extreme cases smoothly which are the
Newton method(when m0) and the steepest
decline method(when m).

4 Conclusions
Train the network by five kinds of BP
algorithm and the result shows that for the part
of the rapidity of convergence, the L-M
algorithm is the fast and its precision is the
Fig.2 Training curve of L-M algorithm highest, but other algorithms are not so good
Table 1 the comparison of training effect in as the L-M algorithm relatively. L M
different BP algorithms algorithm is fit to some trainings which have a
biggest great quantity of samples.
Training BP training convergence In addition, the reliability of digital
function algorithm times / precision recognition using neural network can be
actual gotten by using hundreds of vectors that have
training random noise. If a higher recognized precision
times is required, it can lengthen the time of network
Gradual 10000/ 0.0999982 training and make the training error precision
traingd descendent 6842 higher or increase the numbers of the neurons
BPalgorithm in hidden layer. Otherwise, it can enhance the
Gradual resolving power of input vectors, for example,
traingda descendent 5000/ 0.0971766 adopting 1616 lattices and so on.
self-adaptw/l 130
BPalgorithm References:
Gradual 20000/ 0.0999995
[1] Ying Liandong.The Design and Application of
traingdm descendent 18373 BP Neural Network [J]. Information
w/momentum Technology,2003,27(6):18-20.
[2] NG S C, CHEUNG C C,LEUNG S H. Fast
BPalgorithm Convergence for Back - Propagation Network
Gradual with Magnified Gradient Function[ J ].IEEE,
2003, 9 (3) : 1903 - 1908.
traingdx descendent 5000/ 0.0957755 [3] He Qingbi, Zhou Jianli. The convergence and
w/momentum 156 improvements of BP neural network[J]. Journal
Of ChongQing Jiao TTong University,
and self-adapt 2005,24(1):143-145.
BPalgorithm [4] Fan lei, Zhang Yuntao, Chen Zhenjun.
Application of Improved BP Neural Network

-5-
 http://www.paper.edu.cn

Based onMatlab[J]. Journal of China West

Normal University(Natural Sciences), 2005,
26(1):70-73.
[5] Zhang Dexi, Bi Yuhua. The Application of
MATLAB in Pattern Recognition[J]. Journal Of
XuChang Teachers COLLEGE,
2002,21(5):43-46.
[6] Yang Zhongjin, Shi Zhongke. Architecture
Optimization for Neural Network[J]. Project and
Application Of Computer, 2004,25:52-53.

-6-