Intelligent learning control for a class of nonlinear dynamic systems

W.G.Seo, B.H.Park and J.S.Lee

Abstract: A controller is presented which guarantees system stability by using a feedback controller coupled with an intelligent compensator, and achieves precise tracking by using a set of iterative learning rules. In the feedback plus intelligent controller unit, the feedback control part stabilises the overall closed-loop system and keeps its error bounded, and the intelligent compensator estimates and compensates for the nonlinear part of the system, thereby keeping the feedback gains reasonably low in the feedback controller. In the iterative learning controller, a simple learning control rule is used to achieve precise tracking of the reference signal and a parameter learning rule is used to update the parameters of the intelligent compensator, thereby identifying the uncertain nonlinearity as closely as possible.

1 Introduction

There has been a substantial amount of research work [l61 in the development of various iterative learning control schemes. The concept of iterative learning control has been introduced to improve tracking performance in an iterative manner by attempting to execute the desired motion repeatedly. The control effort in each attempt is improved by using the tracking error signals obtained from the previous trial. In contrast with the time adaptive controller that updates its control law in the time domain, the iterative learning controller updates its control law in an iterative manner and is able to achieve excellent transient performance in the closed-loop system. Arimoto et al. [I] proposed a D-type general iterative learning controller for a class of nonlinear systems whose input and output gain matrices are of linear time-invariant form and Kuc et al. [2] proposed a control scheme that consists of a feedback compensator and a P-type learning controller. Park e t al. [6] presented an efficient adaptive learning controller for robot systems using the fact that they are linearly parametrisable. In their control scheme they updated not only the learning control input but also the parameter estimates. It is not applicable, however, to general nonlinear systems because they cannot be described by a convenient linear parametrisation form in general. To overcome the problem, an intelligent representation technique is introduced in this paper where the nonlinear compensation terms are approximated by intelligent systems, each of which can be formulated as a linearly parametrisable form. In this context, an intelligent svstem is either a linearly
IEE, 1999 IEE Proceedings online no. 19990520 DOI:10.1049/ip-cta:19990520 Paper first received 29th May 1998, and in revised form 19th November 1998 W.G. Seo and J.S. Lee are with the Department of Electrical Engineering, Pohang University of Science and Technology, Pohang 790-784, Korea E-mail: wongi@tau.postech.ac.kr, jsoo@postech.ac.kr B.H. Park is with the Instrumentation and Control Research Team Technical Research Lab, Pohang Iron & Steel Company, Pohang P. 0. Box 36, Korea E-mail: pc553990@smail.posco.co.kr
IEE Proc.-Control Theor?,Appl., Vol. 146, No. 2, Murch 1999

parametrisable fuzzy logic system that is formed by using fuzzy basis functions or an artificial neural network that is formed by using radial basis functions. The idea of using an intelligent system to approximate the nonlinear term has been motivated by the universal approximation theorem [7-lo]: any continuous or L2 nonlinear function can be approximated with arbitrary accuracy either by a fuzzy logic system or by an artificial neural network. Consequently, this paper extends the result of Park e t al. [6] and presents an intelligent iterative learning control scheme for a class of general nonlinear systems. Uncertain nonlinear compensation terms are approximated in this control scheme as linearly parametrised forms by using intelligent systems and tracking is achieved by an iterative learning algorithm. In contrast with other learning control schemes [3, 41, it updates in an iterative manner not only the learning control signals but also the parameters of the intelligent system. The following notations and definitions are used throughout the paper. R represents the scalar real space endowed with the L , norm Ilx(t)II = suptE[o, r] Ix(t)I. R" represents the n-dimensional vector space over R endowed Ilx(t)ll, where 1.1 with the L , norm Ilx(t)ll =sup,.[o, denotes the euclidean norm. The n-th derivative of x(t ) with (.) and Iumir2(.) respect to time t is denoted as x(")(t). represent the maximum and minimum eigenvalues, respectively.

Problem formulation

The nonlinear dynamic systems considered in this paper are of the form

where x ( t ) = ( x ( t ) ,x(t), . . . ,x("-')(t)) E R" is the state of the system, and f : R" -+ R and g : R" -+ R. subseFor quent development the following assumptions that hold in many practical systems are imposed on the system (eqn. 1).
I65

Assumption 1: f : R"-+ R and g : R"+ R are Lipshitz That is, there exists l x , &> 0 such that on R". lg(x2(t)) -g(x1(t))I G
& I X Z ( t ) - x,(t)l

If(xz(t))-f(x1(t))l d IflXZ(t)- x , ( t ) l

for all x , ( t ) , x2(t)E R". Assumption 2: For all x(t) E

R, holds that it
Ik(x(t>)l d q t )

gL d g(x(t>) g u >

for some gL, g" > 0 and ag ( t ) 3 0.

Assumption 3: The desired trajectory xd(t) is continuously differentiable up to nth order and there exists an upper bound of ud(t) such that lud(t)l d U/ for all t E [0, r]. x(0) is also assumed to be xd(0). Problem statement: Suppose that x(t) E R" is measurable for t E [0, r]. Then find a sequence of piecewise continuous control input u'(t) E R with which the system trajectory xJ(t)converges, to x d t ) . In other words, given E > 0, find a sequence of uJ(t)such that there exists a positive integer N > 0 such that

gd for g(xdt)); for f ( x d t ) ) ; 2 for (x(t)lOg>; for j . (x(t>l~?>; for (x,(t)IO?); 2 : for j (xqt)e;i);g for . ; (xdt)~~;); for (xdt)lej>; wg for w,(x(t)>; wf for fd* wf(x(t)); wi for w,(xj(t>); w/ for wdxJ(t>);wgd for wg(x(t));wh for wdxdt)). Now, let x i and x d be the resulting states of eqn., 1 due to inputs ' and U d , respecU ' tively. Also, define e = x J - x d and zJ = (a d/dt)"-' eJ = kTeJ, where k = ( k , - , , kn-2,.. ., k l , ko)T, eJ = (ej'n-'J, eJ"-*), . . e j , e;)T and ki . c , . ~ " - l(n~ = - - I)!/ ( n - 1 - i ) ! i ! . d p l - ifor all 1 < i < n - 1. The design parameter k is chosen in such a way that k, s n P 2 . . . + k l s+ko is Hunvitz. The auxiliary signal z J can be rewritten as zj = e-f- kT$ knP3, . . , k , , k,) and Z J = (eJ("-*) $"-3', . , where k - (knP2, . . .,e', ej). Then it follows from eqns. 1 and 2, that

i..

g;(x;(nJ x d ) - (n)

+ (g'

gd)x$) + ' f

-fd

ZZ

UJ

- ud

(3)

which can be rearranged further to

gJzJ

+ G*J -

g@ ;

= UJ - ud -

(d wxd)x$' - (4
-

+p-2

wr,) gJkTe' (4) +

Our approach to this problem is in approximating the nonlinear functions f ( x ( t ) ) and g(x(t)) with intelligent systems and use them to build an intelligent compensator which forms one of the component in the intelligent learning controller.
3

Substituting eqn. 2 into eqn. 4 gives

gjiJ

+ yid* = S

ud - w, '

(5)

Intelligent description of nonlinear systems

As stated in the universal approximation theorem in [7lo], any continuous nonlinear system can be approximated by the intelligent system with arbitrary accuracy in a compact domain. Thus, nonlinear functions g(x(t)) and f ( x ( t ) ) in eqn. 1 can beapproximated by the intelligent systems (x(t) IO:) andf(x(t) IO?) as follows.

where ~ ! = ( [ , ' ( x ~ ) x -)<f(xd)x$', , f r ( x ~ ) [ F ( x d ) ) , e*' = $ ~ Of*') and d = (4- wXd)x$' - wh)-gJkTiJ. The combined intelligent approximation error wi is bounded above as follows. Lemma 1: The intelligent approximation error w/ in eqn. 5 satisfies

(e:,

+ (4

g(x(t))= i ( x ( t ) l e ; > + w g ( m = i,'(x(t>)@; + w,(x(t>>

for some I < 0 and for a l l j d 1. , Pi-ooj See Appendix (section 8.1).
4

and
f(x(tN = h ( t ) I q

+ W f M t ) ) = r ; m q + wf(x(t))

Intelligent learning control method

(2) where ig(x(t)), i f ( x ( t ) ) are, respectively, the intelligent basis function vectors of g(x(t)lO,*, f(x(t)le?), and w x (x(t)), wf(x(t)) are the corresponding intelligent approximation errors. The intelligent basis functions [ ( x (t)) and , ['(x(t)) are represented either by fuzzy logic systems based on fuzzy basis functions or by artificial neural networks based on radial basis functions. The optimum parameter vectors 0; and 07 are defined by

The intelligent learning control law proposed for the system of eqn. 1 is
uJ - cJ

+ U$ + hJ

(7)

where
A .

,J

= yJeJ.uJ . T 3 f

--

=proj(hJ); i
A . A . A

j
.

(81 + k,"&> + Y!$)zJ = ,$-I - p 2 j-1 z

-

QJ

=proj(eJ), 8' = 4;-1

83 yJ-1' z J-1 s

and
r

w;
166

where Og and Ofare some bounded feasible sets of Og and Of, respectively. Therefore it is obvious that lO;l d O,b and 10; I d O,b for some O,b > 0. In the sequel, x is used for g for g(x(t>>; fforf(x(t)>; g; for g(x'(t));f forf(x'(t>>;

e)',

with kg 3 112, PI > 0, f i 2 > 0, 8 3 > 0, and proj (.) is a ' 8 projection operator that limits lh'l and 1 within bounds U; and Bb, respectively. The block diagram of the proposed learning controller is given in Fig. 1, where, the feedback term uj! with nonlinear compensation term cJ stabilises the overall closed-loop system and the learning rule hJ estimates and compensates for the desired control input ud. cJis the approximated intelligent representation of nonlinear term (gJ - gd)x$)) (fJ -fd> whose main effect is to reduce the control load from the feedback

IEE Pror.-Control Theory Appl.. Vol. 146, No. 2, March 1999

Y, (x & ) I

i j

nonlinear

* compensator

Q 2 0 whenever sufficiently large values are chosen for PA. The following theorem establishes that the intelligent learning controller (eqn. 7) keeps the tracking errors bounded and drives them to zero. Theorem 1: Let PI, P 2 , 83 satisfy P I = i(82 P n ) , 82 = pp3 for some p > 0 and 0 < P3 < 2. Also, choose P h and k in such a way that Q = QT > 0, where Q is defined in lemma 3. Then, the controller in eqn. 7 guarantees that

i
K ~ E

feedback

Fig. 1 Structure of proposed control scheme

(ii) (iii)

lim v J ( t )= v(t)
J+'X

lim z J ( t )= 0 for all t E [0, TI

J+ w

term us and maintain the feedback gain Substituting eqn. 7 into eqn. 5 gives gJzJ

PI reasonably low.

+ P1zJ+ k,a,(t)z' + YiYj'z'

I

+ hJ - WJ,

(8)

where OJ = 8 - 8*and hJ = hJ - ud. For the development ' of the theorem the following lemmas are introduced. Lemma 2: Let en(t)= ( ( d / d t ) a)"e(t) and e,(t) = ( ( d / d t ) a)'e(t) for 0 < i < n and n is a positive integer. Then

where d i t ) = $(h"(z) p $ ' ( ~ ) 8 ~ ( z ) ) d zfor all t E [0, r] and j < 1. Proof: See Appendix (section 8.3). Note that, xi can be kept within a compact, set by increasing the feedback gain P2 or by choosing QJ appropriately. The design factor p is a weighting coefficient between h and cJ. If little information is available on the ' system, p must be larger than one to satisfy P2 < P3. Remark 1 : The extension to multivariable case where f(x) ER", E Rnmxl u(t) E R is straightforward g(x) and " by using essentially the same arguments as in the singlevariable case as long asf(x), g(x) are Lipshitz continuous and the positive definite matrix g(x) satisfies gLZ< g(x)<guZ, where A l <A2 is denoted as Amax ( A 1) < Amin(A2).

Proof: See Appendix (section 8.2).

Lemma 3: For a given auxiliary signal zJ and w: as in eqn. 6, there exists a constant f l h such that
5 Simulation results and discussion

In this Section, a simulation is performed on the inverted pendulum model [ 111, whose dynamics are given by where the matrix Q satisfies
4n-1,n-1

9n-1,n-z

. . . 9n-1.0

gsinx,(t) X2(t) =

...

...

mI$(t) cosxl(t) sinx,(t) m,+m i- 4 - mcos2xl(t) (

40,o

3
cos XI ( t ) m, m

m,+m

with its elements

m,+m

where g = 9.8 d s 2 is the acceleration due to gravity, m, is the mass of the cart, m is the mass of the pole and I is the half-length of the pole. The state (x,(t), x2(t)) is (8(t),Q(t)), where 8 is the angle that the pole makes with the vertical line and the control input U is the applied force to the cart. Dynamic parameters are set to m, = 1 kg, m = 0.1 kg and (=0.5 m. Control and learning parameters are set to k = 3 , p = 2.5, kg = 1 and a(t) = IXJ(t)l IxJ(t)l. The learning gains are set to P1= 5, Ph=4.75, /j2 = O S and = 0.2. Finally, the initial states x(0) is set to (d30, 0) and the sampling interval is set to 0.05 s. On the other hand

o3

IEE Proc.-Control Theoiy Appl., Vol. 146, No. 2, March 1999

167

x j ( t ) is limited to (xi(t)J 0.3. In this simulation a fuzzy <

0.15

logic system is used as an intelligent system and the corresponding membership functions are set to

-0.151 0

for i = 1, 2, as shown in Fig. 2. The basis functions for [(.) are 25-dimentional vectors with I( = i . j)th element [ ( r , J ) ( . ) = C ~ ~5 , P ~ , / Cwhere = j,= 1~ ,2I, .*.~5,. , For ~ , ~ i, ~ , . , details of the fuzzy controller, refer to [I 11. To obtain a positive effect of the adaptive fuzzy compensation block, p' must be greater than p3 of cJ.In fact, since e(0) of the fuzzy compensator are set to zero in this simulation, a large value of p3 may render cJ a large disturbance on the system. If partial information on the system is available, it can be reflected on the initial parameter e(0) to compensate for the nonlinear term. Figs. 3 and 5 show the trajectories of xl(t) and xz(t) when the fuzzy compensator is not used, whereas Figs. 4 and 6 show the trajectories ofx,(t) andx2(t) when the fuzzy compensator is used. Fig. 7 shows the summed control input magnitude (E:=, lu(At. k)l), where N = 300 and

8 time$

Fig. 4 Trajectoly of x , (t) with intelligent compensator

0.15

time,s f Fig. 5 Trajectory o x2 (t) without intelligent compensator

-0.15 0

Fig. 2 Fuzzy membership functionsfor 5 0)

8 times

1 0 1 2 1 4

f Fig. 6 Trajectory o x , (t) with intelligent compensator

0.151

-0.151 0

6-

8
tirne,s

10

12

14

I
iteration

f Fig. 3 Trajectory o x I 0) without intelligent compensator

Fig. 7 Summed input power for every iteration

IEE Proc.-Control Theory Appl.. Vol. 146, No. 2, March 1999

168

At = 0.05 s. The simulation results show that the learning controller with the intelligent compensation term enhances the tracking performance without having to increase the corresponding control input power.

9 Appendix
9.1 Proof of lemma 1 Note that

1%
6

- WSd I =

Ig - 2) ( g d - i 2 ) I d Igj - gdl (j -

Conclusions

+ ly - 221
and

An intelligent learning control scheme has been presented for a class of nonlinear dynamic systems. The controller presented extended the result of Park et al. [6] to a class of general nonlinear systems by using an intelligent approximation technique for nonlinear compensation. The controller consists of three control units: feedback control, nonlinear compensation and learning control. The feedback controller provides stability to the system and keeps its state errors within uniform bounds. The nonlinear compensation term reduces the control load from the feedback control term and keeps the feedback gains reasonably small. The learning controller achieves precise tracking without using any output derivative terms that are vulnerable to noise. The nonlinear compensation term is determined by using the intelligent approximation technique for a class of nonlinear systems. Simulation results show that the intelligent compensator enhances tracking performance without increasing input power. As shown in Fig. 7, the control input power can be reduced with a properly chosen intelligent compensator. The developed controllers are rather complex, which normally require much computation power and memory for the generation of real-time control input. Fortunately, the CPU power and memory become less of a problem these days with the aid of semiconductor technology and fast processors becoming available at low price.

I$

- wJdl =

l ( f -?) +I? -21

-fill < fI

-fdl

Then assumption 1 and the mean-value theorem show that

and

Then it follows that

where
7

Acknowledgments

This work was supported by the Korea Science and Engineering Foundation through the Automation Research Center at Postech. The authors would like to thank the anonymous reviewers for the constructive comments on the paper.

and

8 References
1 ARIMOTO, S., KAWAMURA, S., and MIYASAKY, E: Bettering operation of robots by learning, 1 Robot. Syst., 1984, pp. 123-140 2 KUC, T. Y., LEE, 3. S., NAM, K., An iterative learning control theory for a class of nonlinear dynamical systems, Autornatica, 1992, 28, pp. 1215-1 22 1 3 AHN, H. S., CHOI, C. H., and KIM, K. B.: Iterative learning control for a class of nonlinear systems, Automatica, 1993, 29, pp. 1575-1578 4 JANG, T. J., CHOI, C. H., and AHN, H. S.: Iterative learning control in feedback systems, Automatica, 31, 1995, pp. 243-248 5 CHIEN, C., and LIU, J.: A P-type iterative learning controller for robust output tracking of nonlinear time-varying systems, Int. 1 Control, 1996, 64;(2), pp. 3 19-334 6 PARK, B. H., KUC, T. Y., LEE, and JIN, S.: An adaptive learning control of uncertain robotic systems, Int. 1 Control, 1996, 65, (5), pp. .. 725-744 7 WANG, L. X.: Stable adaptive fuzzy control of nonlinear systems, IEEE Trans. Fuzzy Syst., 1993, 1, (2), pp. 146-155 8 WANG, L. X.: Adaptive fuzzy systems and control: Design and stability analysis (Prentice-Hall, Englewood Cliffs, NJ, 1994) 9 POGGIO, T., and GIROSI, E: Networks for approximation and learning, Proc. IEEE, 1990, pp. 1481-1467 10 KRYZAK, A., and PAWLAK, M.: The pointwise rate convergence of the kernel regression estimate, 1 Stat. Plan. Inference, 1987,16, (3), pp. 159-166 11 WANG, L. X.: A course in fuzzy systems and control (Prentice-Hall, Englewood Cliffs, NJ, 1997)
IEE Prnc.-Control Theory Appl., Vol. 146, No. 2, March 1999

11 a (i(x)la xi1 and 18 [F(x)la xII can be calculated off-line 1 once the intelligent functions are determined. Hence I, =I x ~ ~ ~ Y , rf+gu Ikl is obtained. U

9.2 Proof of lemma 2 The lemma is proved by mathematical induction. For n = 1, it holds that

1;(rG)2+

a2e2(z))dz ae2(t)

which is eqn. 9. For n = m, suppose that

169

If the controller of eqn. 7 is used the second term in eqn. 14 disappears and it follows that Ad 6

[((pi
0
-

- 2p1p2)zJ2 2p2zJgJZJ 2p2kgugzJ2 -

2p2wJ,z')dz

<

- p2zj2gJ

JI

(p2(1- 2kg)ugzJ2

+ (pi -

2/31p2)zJ2

+ 2P2lwJ,IIzJl)dz
(by integration by parts and assumption 3)

(by lemma 2) Since eqn. 9 holds for n = m 1, it follows by mathematical induction that it holds for all integer n 2- 1.
-. Let h = hJ - udandtIJ= 8 - 8*. Then A i J := iJ+'' ='+I $ = -p2zJ and AOJ := e' - OJ = -p3Y/zJ. If CJ is
=j

9.3 Proof of theorem I

-. -. - ,

n-1 n-1 k=O i=O

defined as $(t) = $($2(z)

+ pgT(z)$(z))dz,it follows that

Zj+]T

< <

p2zjzgi - 2P2 /l(dTQeJ)dz

(by lemma 3)

A d = vj+l(t)- d(t)< Vj+'(t) - d ( t )

=
=

S:,

-i j 2

.(8

-p2zJ2gJ

'

- ,jjTfij))d,

+ Y,jYCZJ- Y,j8J)+ p(p:zJ2&JYiT- 2p,zjqJB'))dz

I I

(Ah"jz + 2 A h j 2 + p(AB"j'A6J + 2A@'6J))dz

(&zJ2- 2fi2zj(gJzJ plzJ kgugzJ wJ,

Thus v,' 2 vi - d+ > p2gLzJ2 a l l j 2 1, which proves for that IzJI is bounded. Next, since vi is positive definite and monotonically decreasing, v J converges to some .positive function and this implies that A vJ + 0 and zJ + 0 as j + CO [6]. Since ki (i= 0,. . ., n - 1) are coefficients of a Hurwitz polynomial, zJ = 0 ensures the global convergence of x to x d asymptotically.

170

IEE Proc.-Control Theory Appl., Vol. 146, No. 2, March 1999