Laboratory work 6 Hybrid Intelligent Systems

Neural System Identification

Goal
Learn to design MLP and RBF for dynamic system identification, using MATLAB Neural Network
Toolbox.

Theory Overview
Given a predefined ANN structure, the neural identification becomes in fact a parametric identification
with the following particularities:
the parameters of the model are the neural parameters;
the neural parameters have no real significance;
the parameters are optimized via the training algorithm.
ANNs allow the identification according to NARX-NARMAX templates.
Let us consider a SISO dynamic system described by:
y ( k ) = f ( y ( k 1),...., y ( k n), u ( k ), u ( k 1),......., u ( k m)) , where
y (k ) plant output at time instant k,
u (k ) plant input at time instant k.
In order to be able to mimic the behavior of a dynamic plant, the static ANNs should be combined with
external dynamic blocks.

Identification schemes for feedforward ANNs

Input
System

Output
+

-1

q-1

ANN

-1

q-1

Estimated output

External delays

Fig. 1 Series-parallel scheme containing output links

The series parallel schemes (recommended for this laboratory work) are outlined in Fig. 1. Their
main advantage is related to the fact that usually they lead to a stable learning.

ANN inputs are current and lagged values of plant input and output; the lagged values are provided
by external dynamic blocks.
Hence, the resulted neural models (named timed ANNs) are compliant with:
)
y ( k ) = f ( y ( k 1),...., y ( k n), u ( k d ), u ( k d 1),......., u ( k m )) , where
d - dead time
m, n - maximum input/output delays.
For example, d = 0, n = 2, m = 1 lead to the ANN portrayed in Fig. 2.

y(k-1)
y(k-2)

RNA
u(k-1)

y RNA (k )

u(k)

Fig. 2 ANN inputs and outputs (for d = 0, n = 2, m = 1 )

In this case, the training/validation samples should be compliant with the following template:
u (k )
u (k 1)
and d RNA (k ) = y (k ) .
u RNA (k ) =
y (k 1)

y ( k 2)
In order to appropriately illustrate the properties of the dynamic system, the input used for
generating the training/validation data sets should be persistent. In this attempt, step input, white
noise, SPAB are recommended. It should be noticed that model validation requests data sets
different than the one used for training.
To diminish the risk of ill conditioning, the elements of the training/validation samples
(u RNA , d RNA ) could be scaled between [-1, 1]:
u min(u )
y min( y )
u scalat = 2
1 , y scalat = 2
1.
max(u ) min(u )
max( y ) min( y )
Convenient values of m, n, d could be chosen by trial and error. The designer must also select an
appropriate sampling period, using available apriorical information about system frequency
spectrum.

Problem Set
Problem 1
Let us consider a SISO dynamic system described by the transfer function:

G( s) =

s +1
s 2 + s +1

Design an MLP and an RBF able to approximate its behavior.

Firstly, assume that d = 0, n = 2, m = 1 .
MLP should be trained with traingdm (BP with momentum) or traingd (BP without momentum).
Convenient values for the following parameters must be selected by trial and error:
- number of hidden neurons, GOAL, SPREAD for RBF;
- number of hidden neurons, number of epochs, learning rate for MLP.
The comparative analysis should reveal:
o the approximation performances of the trained ANNs obtained for the training and the
validation data sets e.g., in terms of the squared output error:
Yrna_a=sim(net,U); % training
sum((D-Yrna_a).^2)
Yrna_a=sim(net,Ut); % validation
sum((Dt-Yrna_t).^2)
o the number of neural parameters.
For RBF only, consider also d = 0, n = 1, m = 1 and d = 0, n = 3, m = 4 . Comment the results.

Recommendations:
The system admits the cutting frequency wo = 1 , hence the sampling period should be compliant with
T < 1 / 2 = 0.5 One chooses T = 0.05.
In order to obtain the same results at independent algorithm executions, one can initialize the random
generator as indicated below:
randn(seed,0);
rand(seed,0);
The training data set can be generated as follows:
T=0.05; %sampling period
NNA=250; %number of training samples
TN=1;
% for additional decimation of samples
u=2*randn(NNA,1)-1;%white noise, with mean 0 and spread 1;
% or step input
sys=tf([1 1],[1 1 1]);
%the model of the system
time=[0:T:(length(u)-1)*T]; %time instants used for data sets generation
y=lsim(sys,u,time); %plant output - simulation
u=2*(u-min(u))/(max(u)-min(u))-1;%scaled within [-1, 1];
y=2*(y-min(y))/(max(y)-min(y))-1;%scaled within [ -1, 1];

figure(1);
% training data set with samples: U=[u(k), u(k-1), y(k-1) , y(k-2)], D(k)=y(k);
% training samples collected with step TN
U=[u(3:TN:(length(u))),u(2:TN:(length(u)-1)),y(2:TN:(length(u)-1)), y(1:TN:(length(u)2))]';
D=y(3:TN:(length(u)))';
Time=time(3:TN:length(time));%time vector used for ANN simulations
plot(Time,U(1,:),'m');title ('intrare sistem pentru setul de antrenare').
The experimental trials should also consider NNA=300, TN=2; or NNA=50, TN=1.
For model validation, one can use data sets generated for the following system inputs:
- step;
- ut = sin(0.5t ), w = 0.5 < w0 and ut = sin(2t ), w = 2 > w0 .
- white noise.
Example considering the validation via step input:
NNT=500;
T=0.05;
ut=ones(NNT,1);%step input
timet=[0:T:(length(ut)-1)*T];
yt=lsim(sys,ut,timet);%system output - simulation
yt=2*(yt-min(yt))/(max(yt)-min(yt))-1;%scaling
%Validation data set with samples: Ut=[ut(k), ut(k-1), yt(k-1) , yt(k-2)], Dt(k)=yt(k);
Ut=[ut(3:1:(length(ut))),ut(2:1:(length(ut)-1)),yt(2:1:(length(ut)-1)),yt(1:1:(length(ut)-2))]';
Dt=yt(3:1:(length(yt)))';
Timet=timet(3:1:length(timet));%time vector for simulations
plot(Timet,Ut(1,:),'m');title ('intrare proces pentru setul de testare')

Problem 2
Idem for the SISO linear dynamic system modeled by the transfer function:

G( s) = e s

s+2
.
s 2 + 2 s +1

Additionally, analyze the influence of noisy plant output measuring. The additive white noise will be
synthetically generated, considering a Gaussian distribution with mean 0 and spread 0, 0.02 and 0.1,
respectively.
%noisy training/validation output samples can be built with:
y=lsim(sys,u,time)+randn(length(time),1)*0.02;
%or
y=lsim(sys,u,time)+randn(length(time),1)*0.1;