GEORGIA INSTITUTE

OF TECHNOLOGY

AS SI G NM E N T 5
CONTINUOUS ONLINE IDENTIFICATION WHEN A STEP CHANGE OCCURS IN THE FUNCTION TO BE IDENTIFIED SCOTT L ARISCH ECE 8803A

Assignment 5
ECE 8803A

INTRODUCTION

This assignment extends Assignment 4, but now we will introduce an abrupt change to the function to be identified after 50 seconds of training. The structure of the neural network is shown in Figure 1. The Adaptive Neural Network (ANN) identifier uses a Nonlinear Auto Regressive Moving Average (NARMA) model. Insteasd of a single input, x, along with a bias of 1, now samples of x and previous estimates along with a bias function of 1 are fed to each of the neurons in the hidden layer. Each neuron applies its weights, W, and then evaluates its sigmoidal function. The weights V are applied to the sigmoidal function ouputs and the products are summed to form the neural network estimate, .

Figure 1. Block diagram of the continuous online identifier to be simulated.

As in Assignment 4, the input x is a time varying function given by: x(t) = sin(2t) Initially the function of our plant is: y(t) = 2x2 + 1 After 50 seconds of training the function will be abruptly changed to: y(t) = 2.5(x 0.2)2 + 1.5

We will use on-line training, and so we have to decide on a sampling frequency. Simulation runs will start at t = 0 and sample at every 10 milliseconds. In other words, the sampling frequency will be 100 Hz. We will investigate the effect on system performance by changing the order of the ANN model. Initially simulation runs will be made for a zeroeth order model, i.e., the inputs to the ANN will be x(t) and a bias term of 1. Note that the zeroeth order model was the model investigated in the first three assignments. Then we will change the ANN to a first order system. Here the inputs will be x(t), x(t-1), y(t-1), and a bias term of 1. Table 1 gives a description of the orders of the model.

System Order

0 1 2 3 4 5

x(t) x(t), x(t-1) x(t), x(t-1), x(t-2) x(t), x(t-1), x(t-2), x(t-3) x(t), x(t-1), x(t-2), x(t-3), x(t-4) x(t), x(t-1), x(t-2), x(t-3), x(t-4), x(t-5) y(t-1) y(t-1), y(t-2) y(t-1), y(t-2), y(t-3) y(t-1), y(t-2), y(t-3), y(t-4) y(t-1), y(t-2), y(t-3), y(t-4), y(t-5

Table 1. Model order descriptions for Assignment 5.

As in the previous Assignments, we will use the delta weight update strategy with various learning gains, kl, momentum gains, km, and number of hidden layer neurons. For the delta weight update strategy, first the new weights, Vnew, are calculated as: dVnew = kl * e * DT + km * dVold Vnew = Vold + dVnew Then the new weights, Wnew, are calculated as: dWnew = kl * e * Vnew * d * (1-d) * X + km * dWold Wnew = Wold + dWnew

The output tracking error is calculated as: e = y VT * d We will plot the tracking error, e, versus the number of epochs. The mean square error (MSE) will be calculated for each combination of number of neurons in the hidden layer, learning gain, momentum gain, and identifier order. Note that, since there is only one output for this system, the mean square error is calculated by summing the square of the tracking error at each epoch and then dividing by the total number of epochs, MSE = (1/epochs)

(y V

* d)2

S I M U L AT I O N C A S E S

Table 2 identifies the cases that were simulated. For example, Case 1 involved setting the number of neurons in the hidden layer to 5, using a learning gain of 0.1, and using a momentum gain of 0.005 for a zeroeth order model. At the beginning of each run the weights W and V were initialized to the same sets of random values, Wo and Vo, to maintain consistency from one run to the next. Initial weight values can be found in the attached MATLAB code listing.

Case

Neurons in Hidden Layer n_hidden

Identifier Order

Learning Gain kl

Momentum Gain km

1 2 3 4 5 6 7 8 9 10 11 12

5 5 5 5 5 5 5 5 5 5 5 5

0 0 1 1 2 2 3 3 4 4 5 5

0.1 0.2 0.1 0.2 0.1 0.2 0.1 0.2 0.1 0.2 0.1 0.2

0.005 0.2 0.005 0.2 0.005 0.2 0.005 0.2 0.005 0.2 0.005 0.2

Table 2. Simulation cases for Assignment 5.

S I M U L AT I O N R E S U LT S

The following plots show the results of our simulation runs for the various combinations of neurons in the hidden layer, learning gains, momentum gains, and identifier order. The weights W and V were always initialized to the same sets of random values, Wo and Vo, to maintain consistency from one run to the next. A plot for each case showing the results of the tracking and the tracking error versus number of epochs is given in the following figures. Each plot has three subplots. In the first (upper) subplot the actual output (in red) is plotted along with the estimated output (in blue) as a function of time from 49 to 51 seconds. Inspection of this subplot reveals whether or not the neural network was able to approximate the input and how well. We can make qualitative evaluations of how well the tracker performs based on amplitude and phase differences between the two traces. The second (middle) subplot shows the tracking error (the simple difference between actual and estimate) as a function of time, again from 49 to 51 seconds. This plot yields a more quantitative evaluation of how the tracker performs. The third (bottom) subplot shows the tracking error (the simple difference between actual and estimate) as a function of time at every sampling instant from 0 to 80 seconds. This plot also yields a more quantitative evaluation of how the tracker performs. The title of the second subplot includes the mean square error of the tracking error.

Figure 2. Simulation results for Case 1 (0th Order, kl = 0.1, km = 0.005, and n_hidden = 5).

Figure 3. Simulation results for Case 2 (0th Order, kl = 0.2, km = 0.2, and n_hidden = 5).

Figure 4. Simulation results for Case 3 (1st Order, kl = 0.1, km = 0.005, and n_hidden = 5).

Figure 5. Simulation results for Case 4 (1st Order, kl = 0.2, km = 0.2, and n_hidden = 5).

Figure 6. Simulation results for Case 5 (2nd Order, kl = 0.1, km = 0.005, and n_hidden = 5).

Figure 7. Simulation results for Case 6 (2nd Order, kl = 0.2, km = 0.2, and n_hidden = 5).

Figure 8. Simulation results for Case 7 (3rd Order, kl = 0.1, km = 0.005, and n_hidden = 5).

Figure 9. Simulation results for Case 8 (3rd Order, kl = 0.2, km = 0.2, and n_hidden = 5).

10

Figure 10. Simulation results for Case 9 (4th Order, kl = 0.1, km = 0.005, and n_hidden = 5).

Figure 11. Simulation results for Case 10 (4th Order, kl = 0.2, km = 0.2, and n_hidden = 5).

11

Figure 12. Simulation results for Case 11 (5th Order, kl = 0.1, km = 0.005, and n_hidden = 5).

Figure 13. Simulation results for Case 12 (5th Order, kl = 0.2, km = 0.2, and n_hidden = 5).

12

TA B U L AT E D R E S U LT S

The plots for the simulation cases show typical results with two different combinations of learning gain, momentum gain for each identifier order. Note that the 0th Order simulations correspond to the simulations run for Assignment 3. The mean square error was recorded for each case. The results are given in Table 3.

Cases

Neurons in Hidden Layer n_hidden

Identifier Order

Mean Square Tracking Error (kl = 0.1, km = 0.005)

Mean Square Tracking Error (kl = 0.2, km = 0.2)

1 and 2 3 and 4 5 and 6 7 and 8 9 and 10 11 and 12

5 5 5 5 5 5

0 1 2 3 4 5

0.084708 0.036515 0.023515 0.013883 0.021601 0.016454

0.016711 0.011746 0.010817 0.014667 0.025726 0.024176

Table 3. Results for simulations tabulating mean square tracking error.

All simulations were run for 80 seconds of simulation time. From this table we see that mean square tracking error decreases when the identifier order is increased from 0 to 2. A zeroeth order system provides only the current value of x and the bias value of 1 as inputs to the ANN. Increasing the order to 2 provides not only previous values of x, i.e., x(k-1), x(k-2), but also previous values of the estimates, (k-1), (k-2). These previous values allow the ANN to more accurately predict the next value. For example, with the previous value x(k-1) as well as x(k), a simple differencing yields the rate of change (or the velocity) of x. Likewise, two previous values x(k-2) and x(k1) as well as x(k) can be used to determine the rate of change of the velocity (acceleration). With higher orders more accurate prediction can take place. However, increasing the order of the identifier beyond 2 tends to increase the mean square tracking error. For example, a fourth order identifier provides previous values of x, i.e., x(k-1), x(k-2), x(k-3), x(k-4), and previous values of the estimates, (k-1), (k-2), (k-3), (k-4). Increasing the number of inputs to the ANN now has a deleterious effect on system performance. This is most likely due to the hidden layer having only five neurons. The higher order systems are presenting more information than five neurons can manipulate. It is expected that increasing the number of neurons in the hidden layer would improve the performance of the higher order identifiers.

13

Figure 14 shows the results for Case 12, but now from 49 to 57 seconds. While the identifier is able to track the abrupt change, it has difficulty maintaining its tracking. Periodically, the tracking error spikes and then settles down again. The bottom subplot in Figure 14 shows that this phenomenon continues for the full 80 seconds of simulated time.

Figure 14. Simulation results for Case 12 (5th Order, kl = 0.2, km = 0.2, and n_hidden = 12) showing the estimate and tracking error from 49 to 57 seconds.

14

C O N C LU S I O N S

For this assignment we used the neural network as a continuous online identifier. After 50 seconds of training, the function of the plant was abruptly changed. We investigated various combinations of system order, learning gain, and momentum gain for both five neurons in the hidden layer. Improvements in performance were found when increasing the order of the identifier from 0 to 2. Increases in order of the identifier beyond 2 tended to decrease system performance (increase mean square tracking error). It is strongly suspected that this was due to having only five neurons in the hidden layer. It is expected that increasing the number of neurons in the hidden layer would improve the performance of higher order identifiers. Additional simulations would have to be performed to confirm this.

15

M AT L A B S O U R C E CO D E

%ANN DEFINITION n_hidden = 5; n_out = 1; %LEARNING GAIN kl = 0.2; %MOMENTUM GAIN km = 0.2; %USE SAME RANDOM NUMBER STREAM EVERY TIME FOR Wo AND Vo Wo = [ 0.9003, -0.0069, 0.6770, -0.2410;... -0.5377, 0.7995, 0.1361, 0.6636;... 0.2137, 0.6433, -0.2592, 0.0056;... -0.0280, 0.2898, 0.4055, 0.4189;... 0.7826, 0.6359, 0.0931, -0.1422 ] Vo = [ -0.4312, -0.1530 ] W = Wo; V = Vo; %BACKPROPAGATION ALGORITHM max_epoch = 8000; MSE_umbral = 1e-6; epoch = 1; MSE(epoch) = 1; dV = zeros(1, n_hidden + 1); dW = zeros(n_hidden, 12); %USE SAME RANDOM NUMBER STREAM EVERY TIME FOR X %RandStream.setDefaultStream(RandStream('mt19937ar','seed',2.3232e+005 )); %Initialize inputs x = 0.0; xm1 = 0.0; xm2 = 0.0; xm3 = 0.0; xm4 = 0.0; xm5 = 0.0; 0.8436, -0.6541, 0.4764, 0.9595, -0.6475, -0.4571, -0.1886, -0.4953, 0.8709, 0.7515, -0.0616, -0.7222, -0.9630, -0.5945, 0.6428, -0.6026, -0.1106, 0.2076, 0.2309, -0.4556, 0.5839, -0.8704, 0.6924, 0.7873, 0.0503, -0.8842, -0.5947, -0.2943, 0.3443, 0.6263, 0.6762, -0.9803, 0.9767, -0.3908, 0.4936, -0.6207, -0.1098, -0.6131, 0.8636, 0.3644, -0.0680, -0.3945, -0.1627, 0.1656,

16

y = ym1 ym2 ym3 ym4 ym5

0.0; = 0.0; = 0.0; = 0.0; = 0.0; = 0.0;

%while ((epoch < max_epoch + 1) & (MSE(epoch) > MSE_umbral)) while (epoch < max_epoch + 1) %x(1,1) = 2 * rand(1,1) - 1; %y = 2 * x.^2 + 1; %Update ANN inputs xm5 = xm4; xm4 = xm3; xm3 = xm2; xm2 = xm1; xm1 = x; x = sin(2*pi*epoch/100.0); ym5 = ym4; ym4 = ym3; ym3 = ym2; ym2 = ym1; ym1 = y; if ( epoch <= 5000 ) y = 2 * x^2 + 1; else y = 2.5 * ( x - 0.2)^2 + 1.5; end; % Save the actual output for plotting yp(epoch,1) = y; %input threshold is added X = [1 x xm1 ym1 xm2 ym2 xm3 ym3 xm4 ym4 xm5 ym5]; a = W * X'; %Sigmoidal function is used: d = 1./(1 + exp(-a)); %The output error is calculated: %e = y - V*[1;d]; % Save the estimated output for plotting Yp(epoch,1) = V*[1;d]; e = y - Yp(epoch,1); % Save tracking error for plotting TE(epoch,1) = e; %Weight Update (threshold in the output is added): dV = kl*e*[1;d]' + km*dV; V = V + dV; for k=1:n_hidden;

17

dW(k,:)=kl*(e*V(k+1))*d(k)*(1-d(k))*X + km*dW(k,:); end; W = W +dW; %Sum of Square Errors: %xe(epoch,1) = x; %ye(epoch,1) = y; %if(epoch % Xe = % Ye = %else % Xe = % Ye = %end; < 500), [ones(epoch,1) xe]; ye; [ones(500,1) xe(epoch-499:epoch,1)]; ye(epoch-499:epoch,1);

%Ae = W*Xe'; %De = 1 ./(1 + exp(-Ae)); %Yest = (V*[ones(1,length(Xe(:,1))); De])'; %E = Ye - Yest; %MSE(epoch+1) = mean(E.^2); %MSE(epoch) = mean(E.^2); epoch = epoch + 1; end; % Save final values of weights Wf = W; Vf = V; % Find the mean square of the tracking error MSEm = mean( TE.^2 ); %PLOTS %N=100; N = max_epoch; %xp = [-1:2/N:1]' xp = [0.01 : 0.01 : max_epoch/100.0]'; %yp = 2*xp.^2 + 1; %yp = 2 * sin( 2*xp.^2 + 1 ); %Xp=[ones(N+1,1) xp]; %Ap = W*Xp'; %Dp = 1./(1 + exp(-Ap)); %Yp = (V*[ones(1,N+1);Dp])'; MSEp = min(MSE); MSEs = num2str(MSEp); % Plot from 49 to 51 seconds subplot(3,1,1); plot(xp(4900:5100),yp(4900:5100),'r',xp(4900:5100),Yp(4900:5100)); title(strcat('Fifth Order Model, y(x)=2x^2+1 (red) and Estimated output (blue), n_h_i_d_d_e_n = ', num2str(n_hidden), ', k_l = ', num2str(kl), ', k_m = ', num2str(km) )); xlabel( 'Time (sec)' ); ylabel('y(x)=2x^2+1,y_e_s_t_i_m_a_t_e_d');

18

% Plot tracking error from 49 to 51 seconds subplot(3,1,2); plot(xp(4900:5100), TE(4900:5100), 'r'); title('Tracking Error'); xlabel('Time (sec)'); ylabel('Tracking Error'); % Plot Tracking Error subplot(3,1,3); title( strcat( 'Tracking Error, Mean Square Error = ', num2str(MSEm) ) ); xlabel('Time (sec)'); ylabel('Tracking Error'); hold on; plot(xp(1:8000), TE(1:8000), 'r'); hold off; MSEm

19