Information Sciences and Computer Engineering, Vol. 2, No.

1, (2011) 712
International Journal of
Information Sciences and Computer Engineering
j our nal homepage: http://www.ijisce.org
Recurrent Neural Networks (RNNs) Controller For Dynamical System
Yousif I. Al-Mashhadany

Electrical Engineering Dept., Al-Anbar University, Baghdad, Iraq

Abstract The ability of Articial Neural Network (ANN) for controlling dy-
namical systems is presented. The structure of a neural controller based
on model reference adaptive control (MRAC) is available. The Levenberg-
Marquardt back propagation (LMBP) is used in the learning of Locally Re-
current Neural Networks (LRNNs) to identify the plant and as a neural con-
troller. This method has the ability to capture the nonlinearity and overcome
the problems of dynamic systems. The parameters of the neural controller
are adjusted with time by using the error signal between the output of the
model reference and output of the system through the adjusting mechanism
based on LMBP algorithm.
A simple airplane channel at a certain freezing point is used as an ex-
ample in this paper. More satisfactory results are obtained, with the locally
recurrent neural network (LRNN) controller when compared with a classical
controller for dynamical system.
Keyword: Locally recurrent neural network(LRNN), Levenberg-Marquardt
back propagation (LMBP), model reference adaptive control (MRAC)
1. INTRODUCTION
Model Reference Adaptive Systems (MRAS) have been
adopted by many researchers in controlling dynamical systems.
Such approach only requires the input and output measurements
of the system and is, thus, congruous for plants where mathemat-
ical models are unavailable or dicult to obtain. In addition to
this advantage, the stability of the system is somehow assured
through the convergence of both the states and parameters of the
plant and the reference model In MRAS, the controller parame-
ters are directly adjusted to reduce some norm of the output error
between the plant output and the desired reference trajectory [1].
Neural Network systems, on the other hand, are highly com-
pelling for controlling nonlinear dynamic systems with unknown
parameters. The integration of neural methods eectuates an ex-
cellent learning and exible knowledge representational capabil-
ity. The data driven neuro systems meliorate heuristic procedures
or expert knowledge in designing the neuro control rules which
has been a drawback in conventional neural systems. Moreover,
the neural network is suitable to be used in solving nonlinear

Corresponding author:
Email addresses: yousif unv2009@yahoo.com, Ph: +60 104217541
identication and control problems involving complex plants es-
pecially when forming a mathematical model of the system is
tedious or not possible. The Levenberg-Marquardt back propa-
gation algorithm is used for learning the parameter identication
which provides the consequent parts of the rules [2].
In this paper, a Neural Model Reference Adaptive Controller
(NMRAC) is proposed for nonlinear systems with unknown pa-
rameters. The control strategy used to dene the adaptation law
is based on the tracking error between the actual plant output and
target output which is the response of the reference model. Then
the tuning of the parameters of network is based on the standard
delta rule or steepest descent algorithm to minimize the track-
ing error. This algorithm is preferred since the weights update
is governed by the rst derivative of the error, and thus produces
a faster rate of convergence, consistent training and no getting
stuck in local minima. The system structure identication mech-
anism includes the technique for input data partitioning, auto-
matic generation of rules and tuning of its parameters based on
observed input-output data of a reference signal.
The use of an articial neural network as an adaptive con-
troller to control a simple dynamical system. Psaltis et. al [3]
and G.W. Irwin [4] who proposed four dierent learning archi-
tectures based on the neural network approach: indirect learning,
general leaning, direct learning and specialized learning architec-
tures. Fig 1 represents model reference adaptive controller.

(yr)
N.N
controller
+
-
O/P
(y)

Error To
update
N.N.
Input
Signal

Plant
System
idencaon
Model Reference
u
y
Fig. 1. Model reference neural network controller.
8 Al-Mashhadany/Information Sciences and Computer Engineering, Vol. 2, No. 1, 2011
Two main methods exist for providing a neural network with
dynamic behavior: the insertion of a buer somewhere in the
network to provide an explicit memory of the past inputs, or the
implementation of feedbacks. As for the rst method, it is built
on the structure of feed forward networks where all input sig-
nals ow in one direction, from input to output. Then, because a
feed forward network does not have a dynamic memory, tapped-
delay-lines (temporal buers) of the inputs are used.
The buer can be applied at the network inputs only, keeping
the network internally static as in the buered multilayer per-
ception (MLP) or at the input of each neuron as in the MLP
with Finite Impulse Response (FIR) lter synapses (FIRMLP).
The main disadvantage of the buer approach is the limited past-
history horizon which needs to be used in order to keep the size
of the network computationally manageable, thereby preventing
modeling of arbitrary long time dependencies between inputs and
outputs. It is also dicult to set the length of the buer given a
certain application [5].
The most general example of implementation of feedbacks in
a neural network is the recurrent neural network constituted by
a single layer of neurons fully interconnected with each other or
by several such layers. Because of the required large structural
complexity of this network, in recent years growing eorts have
been propounded in developing methods for implementing tem-
poral dynamic feedback connections into the widely used multi-
layered feed forward neural networks.
Recurrent connections can be added by using two main types
of recurrence or feedback: external or internal. External recur-
rence is obtained for example by feeding back the outputs to the
input of the network, while the internal recurrence is obtained
by feeding back the outputs of neurons of a given layer in inputs
to neurons of the same layer, giving rise to the so called Locally
Recurrent Neural Networks (LRNNs) [6].
LRNNs are widely acknowledged as an eective tool that can
be employed by a wide range of applications that store and pro-
cess temporal sequences. The ability of LRNNs to capture com-
plex, nonlinear system dynamics has served as a driving motiva-
tion for their study. LRNNs have the potential to be eectively
used in modeling, system identication, and adaptive control ap-
plications.
The proposed RNN learning algorithms rely on the calculation
of error gradients with respect to the network weights. What dis-
tinguishes recurrent neural networks from static, or feed forward
networks, is the fact that the gradients are time dependent or dy-
namic. This implies that the current error gradient does not only
depend on the current input, output, and targets, but rather on its
possibly innite past.
2. System Identication
The N.N. is used to identify the system based on the network
shown in Fig 2, which represents the LRNNs [6]: It consist of an
i/p Layer (I), hidden Layer (H) and output Layer (O). The weight
matrix consists of WIBB
mxiBB
and WOBB
nxmBB
, the characteris-
tic of the network is that the units in hidden layer are connected
with themselves and each other. However, the information in
such units at one time can only be transferred until being replaced
by new data after nite steps, i.e. the units in hidden layer only
memorize the information for nite time which is the key dier-
ence from the fully recurrent BP. Thus, the network here is called
nite recurrent back propagation network. Hence at the time (t)
the input to the (ith) hidden units is:
AHi (t) =
I

k=1
WT
ik
I
k
+
m

k=1
WH
ik
f (AH
k
(t 1));
i = 1, 2 .....m
_

_
(1)

Fig. 2. Scheme of LRNN with one hidden layer [6].
The activation function (f) is a sigmoid, that is:
f (x) =
1
2
+
1
_
1 + exp(x)
_ (2)
The output of the network is a weighted sum of the hidden unit
o/ps:
O
i
(t) =
m

k=1
WO
ik
f (AH
k
(t)) ; i = 1, 2, ....n (3)
The net is trained by minimization of the total error, which is
evaluated as:
E(t) =
pp

p=1
_
1
2
n

k=1
e
kp
(t)
2
_
, &
E(t) =
pp

p=1
_
1
2
n

k=1
_
T
kp
(t) O
kp
(t)
2
_
_
_

_
(4)
Where; (pp) is the sample length, TBB
kpBB
(t)s are the target
values that the output of the (KBB
thBB
) unit in output layer while
inputting the (pth) sample match at time (t). This can be fullled
by a gradient descent procedure adjusting (w) a long the negative
of w E(t).
The proportion of error for weight updating (correction). The
learning parameter has a profound impact on the performance
of convergence of learning. A plot of the cost function versus
the synaptic weights characterizes the neural network consists of
Al-Mashhadany/Information Sciences and Computer Engineering, Vol. 2, No. 1, 2011 9
a multidimensional surface called error surface. The neural net-
work consists of cross-correction learning algorithm to start from
a n arbitrary point on the error surface (initial weights) and then
move towards a global minima, in step by step fashion. We can
gain understanding and intuition about the algorithm by study-
ing error surfaces themselves the function J(w), show the error
surface. Such an error surface depends upon the task in hand,
but even there are some general properties of error surfaces that
seem to hold over a broad range of real-world pattern recognition
problems.
In this paper the dimension of LRNNs used is (3-11-1) and
the adaptation of weight is achieved by Levenberg-Marquardt
modication algorithm [7]. It is an approximation to Newtons
method. Suppose that we have a function V(x) we want to min-
imize with respect to the parameter vector (x), then Newtons
method would be:
x =
_

2
V(x)
_
1
V(x) (5)
where
2
V(x) is the Hessian matrix and V(x) is the gradient. If
we assume that V(x) is a sum of squares function
V(x) =
N

i=1
e
2
i
(x) (6)
then it can be shown that:
V(x) = J
T
(x)e(x) (7)

2
V(x) = J
T
(x)J(x) + S (x) (8)
where J(x) is the Jacobian matrix:
J(x) =
_

_
e
1
(x)
x
1
e
1
(x)
x
2

e
1
(x)
x
n
e
2
(x)
x
1
e
2
(x)
x
2

e
2
(x)
x
n
.
.
.
.
.
.
.
.
.
.
.
.
e
N
(x)
x
1
e
N
(x)
x
2

e
N
(x)
x
n
_

_
(9)
S (x) =
N

i=1
e
i
(x)
2
ei(x) (10)
For the Gauss-Newton method it is assumed that S (x) 0, and
the update (1) becomes:
x =
_
J
T
(x)J(x)
_
1
J
T
(x)e(x) (11)
The Marquardt-Levenberg modication to the Gauss-Newton
method is:
x =
_
J
T
(x)J(x) + I
_
1
J
T
(x)e(x) (12)
The parameter is multiplied by some factor () whenever
a step would result in an increased V(x). When a step reduces
V(x), is divided by . Notice that when p is large the algorithm
becomes steepest descent (with step 1/), while for small the
algorithm becomes Gauss-Newton. The Marquardt-Levenberg
algorithm can be considered a trust region modication to Gauss-
Newton.
The key step in this algorithm is the computation of the Jaco-
bian matrix. For the neural network mapping problem the terms
in the Jacobian matrix can be computed by a simple modication
to the backpropagation algorithm. The performance index for the
mapping problem is given by [8]. It is easy to see that this is
Fig. 3. Neural model reference adaptive controller and classical controller for
pitch channel of airplane.
10 Al-Mashhadany/Information Sciences and Computer Engineering, Vol. 2, No. 1, 2011
0 50 100
-20
-10
0
10
20
0 50 100
0
5
10
( A )
NN & plant Output Input signal

0 50 100
-4
-2
0
2
x 10
-7
Error
0 50 100
0
5
10
NN Output
me (s) me (s) ( B )
0 50 100 150 200 250
-1
-0.5
0
0.5
1
Reference Model Input
me (s) ( C )

0 50 100 150 200 250
-1
-0.5
0
0.5
1
Reference Model Output (green), Neural Network Output (red)
(D)
me (s)

Fig. 4. A: Input signal and N.N o/p and plant o/p. B: Error signal. C: Reference model input. D: plant o/p and model reference o/p.
equivalent in form to (2), where :
x =
_
w
1
(1, 1)w
1
(1, 2) w
1
(S 1, R)b
1
(1)
b
1
(S 1)w
2
(1, 1) b
M
(S M)
_
T
, and : N = Q S M
(13)
Standard back propagation calculates terms like:

V
w
k
(i, j)
=

S M

m=1
e
2
q
(m)
w
k
(i, j)
(14)
For the elements of the Jacobian matrix that are needed for the
Marquardt algorithm we need to calculate terms like:
e
q
(m)
w
k
(i, j)
(15)
These terms can be calculated using the standard backpropa-
gation algorithm with one modication at the nal layer:

M
= F
M
(n
M
) (16)
Note that each column of the matrix in (16) is a sensitivity
vector which must be back propagated through the network to
produce one row of the Jacobian.
A summary for the Marquardt modication to the back propa-
gation algorithm thus proceeds as follows:
(1) Present all inputs to the network and compute the corre-
sponding network outputs (the standard BP algorithm), and
errors (e
q
= t
q
a
M
q
). Compute the sum of squares of errors
over all inputs (V(x)).
(2) Compute the Jacobian matrix (using eq(9), and eq(10)).
(3) Solve eq(11) to obtain x
(4) Recomputed the sum of squares of errors using : (x + x).
If this new sum of squares is smaller than that computed in
step (1), then reduce by , let (x = x+x).:, and go back to
step (1). If the sum of squares is not reduced, then increase
by and go back to step (3).
(5) The algorithm is assumed to have converged when the norm
of the gradient (eq(3)) is less than some predetermined
value, or when the sum of squares has been reduced to some
error goal [8, 9].
3. Control Architecture
The structure of the controller based on MRAC is shown in
Fig. 1. Its parameters are directly adjusted to reduce some norm
of the output error between the plant output and the desired refer-
ence trajectory. The outputs of neural controller will be entered
to plant [10]. The system used here represents pitch channel in
Al-Mashhadany/Information Sciences and Computer Engineering, Vol. 2, No. 1, 2011 11

0 5 10 15 20 25 30 35 40 45 50
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1
Time (sec)
Amplitude
(rad)
Control signal


Classical controller

0 5 10 15 20 25 30 35 40 45 50
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1
Time (sec)
Amplitude
(rad)

N.N. controller
Fig. 5. Classical controller response and neural adaptive controller.
airplane, with a transfer function [11]:
F

(S ) =
A
1
S
2
+ A
2
S + A
3
B
1
S
2
+ B
2
S + B
3
; (17)
Where, the coecients (A
1
, A
2
. . . ..B
3
) represent the parame-
ters of the transfer function which determine the relation between
the deection of the elevator () and the angle of attack of the air-
plane. The transfer function is calculated at freeing point to get
the numerical expression :
F
zm

(S ) =
- 312.5S
2
- 2884.625S - 2277238.3
S
4
+ 1 11.9393S
3
+ 1964.2416S
2
(18)
The reference model is a second-order dierence equation de-
scribed by:
y(k + 1) = k
a
y(k) + k
a
y(k 1) + (k) (19)
Where the value of (k
a
= 0.0001) [12]. The bounded reference
input is used to simulate both the plant and the reference model.
The training process is based on equations (116) with the same
structure of N.N used in the identication of a system with di-
mension of layers (3-13-1). The set of input to the controller is
[ u(k) e(k)
.
e(k) ]. (Where u(k) is input signal used has the
muilt-level step, e(k) the error signal and
.
e(k) its derivative.
12 Al-Mashhadany/Information Sciences and Computer Engineering, Vol. 2, No. 1, 2011
4. Simulation Results
The simulation of system is achieved using the Matlab soft-
ware Ver.R2008a [13]. Fig 3 represents the Simulink program
for a classical controller for a dynamical system to compare its
results with a neural model reference adaptive controller shown
in Fig 3. The simulation results consist of two parts, the rst is the
identication results. Fig 4(A,B,C&D) represents the procedure
for identication, the network training with random signal with
accepted error around (e-7), the selection of random signal for
learning the network to take every trial that drive the plant with
practice constrain ( such as the limiting of elevator deection and
angle of attack).
The second part is controller results, starting with selecting the
control signal which is represented by a variable level step signal
( similar to practice signal for driven of pitch channel ), the run
time is 50 sec divided by ve intervals with a period of 10 sec.
The next step is to select the model reference, where it must of-
fer to NN controller the optimal similarity with desired response
and with an error that can be controlled without converging. For
the two points above, I use Matlab software to solve MRAS with
LRNN and using the connecting between Matlab/editor program
with Matlab/Simulink to get optimal results for identication of
plant that is rst point in using MRAS in Matlab and it very im-
portant for design where the next stage of controller depends on
knowledge of identify system with plant to made full adaptively
with any input signal (therefore; I made many trail and error to
get error with value 1e
7
for identication) The nal step is build-
ing the adaptation mechanism based on LRNN with LMBP train-
ing algorithm.
Fig 5 represents the two responses for the classical controller
and neural model reference controller based on LRNN. The adap-
tation of NN controller is based on the error between model refer-
ence response and the plant response, this error is entered through
the adaptation mechanism by LRNN training by LMBP algo-
rithm, which gives to this structure high ability to capturing the
desired response.
5. Conclusions
From the results it can be illustrated that the ability of LRNN
with Levenberg-Marquardt back propagation training to control
for dynamic system and overcome the problems of nonlinearity
and its surface learning. The structure of model reference con-
troller has the ability to force the dynamical system to follow the
desired response, while the LRNN has a high speed learning to
get the desired error. This property allows the neural controller to
treat many problems of the system and get the desired response.
By using RNN with MRAS can be treatment the main problems
of dynamic system (the nonlinearity and variations of its parame-
ters), where by using the internal feedback and memory of RNN
can be using the previous states of model as initial condition to
instantaneous state and by using the ability of MRAS structure
for adaptively training with RNN can be overcome the problem
of nonlinearity.
References
[1] S. Sam and C. Wang, Adaptive neural control of uncertain MIMO
nonlinear system, IEEE Transaction on Neural Network, vol. 15,
no. 3, pp. 674692, 2004.
[2] M. S. Lan, Adaptive control of unknown dynamical systems
via neural network approach, in American Control Conference,
pp. 910915, 1989.
[3] D. Psaltis, A. Sideris, and A. A. Yamamura, Amult-layered neural
network controller, IEEE Control System Magazine, pp. 1721,
1988.
[4] G. Lightbody and G. W. Irwin, Direct neural model reference
adaptive control, in IEEE Proceedings on control Theory and its
Application, pp. 3243, 1995.
[5] L. Xiaoou and Y. Wen, Dynamic system identication via re-
current multilayer perceptrons, Information Sciences, vol. 147,
no. 45-63, 2002.
[6] F. Cadini, E. Zio, and N. Pedroni, Recurrent neural networks for
dynamic reliability analysis, Reliability & Risk Analysis: Theory
& Applications, vol. 1, pp. 3042, 2008.
[7] M. T. Hagan and M. B. Menhaj, Training feedforward network
with the marquardt algorithm, IEEE Transaction on Neural Net-
work, vol. 5, no. 6, pp. 989 99, 1994.
[8] Z. Liu and I. Elhanany, A fast and scalable recurrent neural net-
work based on stochastic meta descent, IEEE Transaction on Neu-
ral Network, vol. 19, no. 9, pp. 16521658, 2008.
[9] B. Cannas, G. Celli, A. Fanni, and F. Pilo, Automated recurrent
neural network design of a neural controller in a custom power de-
vice, Journal of Intelligent and Robotic Systems, vol. 31, pp. 229
251, 2001.
[10] M. N. Mahyuddin and M. R. Arshad, Performance evaluation of
direct model reference adaptive control on a coulped-tank liquid
level system, Elektrika, vol. 10, no. 2, pp. 917, 2008.
[11] L. Jack, Spacecraft Aerodynamics. 1960.
[12] H. Husain, M. Khalid, and R. Yusof, Direct model reference adap-
tive controller based-on neural-fuzzy techniques for nonlinear dy-
namical systems, American Journal of Applied Sciences, vol. 5,
no. 6, pp. 769776, 2008.
[13] Matlab for windows, programming language, tech. rep., Math-
work, Var.2009b.
Yousif Ismial Mohammed AL Mash-
hadany (MIEEE , IIE) was born in Bagh-
dad 1973. He received the B.Sc. de-
gree in University of Technology AL-
Rasheed College of Engineering and Sci-
ence / Electrical and Electronic Engi-
neering, Baghdad, Iraq (1995) . M.Sc
degree in Control Engineering from Uni-
versity of Technology AL-Rasheed College of Engineering and Sci-
ence / Electrical and Electronic Engineering, Baghdad, Iraq. (1999)
Ph.Ddegree in Control Engineering fromUniversity of Technology/ AL-
Rasheed College of Engineering and Science / Control Engineering De-
partment, Baghdad, Iraq (2009) . Since 2004, he has been working at the
University of AL-Anbar, Iraq, as a lecturer in the Electrical Engineering
Department. His research interests include biomedical, robotic and con-
trol system. He is now a Post-Doctoral follow research at the Centre of
Research for Power Electronics, Drives, Automation and Control, Uni-
versity of Malaya, Malaysia.