Poles, Zeros, and Feedback: State Space Interpretation

ROGER W . BROCKE?"T,
Abstract-This paper is concerned with the relationships between timeandfrequency domaindescriptions of linear,time-invariant systems and with the evaluation of the effects of feedback on such systems. A new expression for the transfer functionof a system described by a set of first-order differential equations is given; this expression not only relates the poles and zeros to the eigenvaluesof matrices but also makes it possible to compute the transfer function without matrix inversion. The effects of state variable feedback on controllability,observability, and pole-zeroconfigurations are discussed and the effects of feeding back the output and its derivatives are considered. The application of these ideas to an optimal control problem is sketched and methods of extending them to the multiinput, multi-output case are examined.

MEMBER, IEEE

and x is the state vector. It assumed that u and y are is vectors of the same dimension, say nz.,and that x is a vector of dimension n. Particular emphasis will be placed on the case 7% = 1; boldface notation mill not be used for u , y and D in equations which are valid only in thiscontext,and B, C, and D will bewrittenas lo\\-er-case letters. Transforming (1) into the frequency domain, it becomes

(IS - A ) i

BO

+~ ( 0 )

(3)

Space Administration under Contract S o . NsG-496 with the 1 , f . I . T . Center for Space Research, Cambridge, YIass., and in part by the Office of Xaval Research under Contract N O S R 1141(12), SS-stems Research Center, Case Institute Technology, Cleveland, Ohio. of The author is with the Dept. of Electrical Engineering. XIassachusetts Institute of Technology, Cambridge, Mass.

where the circumflex is used todenotetransformed variables. By solving (3) for 2 and using ( 2 ) , it is seen T T CAN NOT be disputed that state space methods t h a t have contributed greatly to modern control theory; y = C(Is - A)-'BO D& C(Is - A)-'x(O). (4) however, a t present the majority design problems of -~ are solved using frequencydomainmethods,andit In theeventthat x ( 0 ) iszero,thefrequencydomain seems doubtful that this situation will change in the relationships bet\\;een 6 and 2 and y simplify to near future. The result is that system engineers must be 2 = ( I S - A)-lBO (5) capable of thinking and communicating both in terms of first-order vector differential representations equation 7 = (C(Is - A)-'B D ) O . (6) and in terms of transfer functions and their related conThis places in evidence the fact that the transfer matrix cepts. In the last few yearsconsiderableprogresshas relating the input to the state vectoris ( I s - A ) - ' B , and been made in clarifsring the connection between these that the transfer matrix relating the input to the output two points of view. T h e basic questions associated with is C ( I s- A)-'B D. the construction of first-order representations have been Since the inverse of (Is- A ) can be expressed as the treated a t length Kalman by [ l ] , Gilbert [ 2 ] , and adjoint of ( I s - A ) times the reciprocal of the determiZadeh and Desoer [ 3 ] ;i n addition, more subtle quesnant of ( I s - A ) , and since the adjoint of ( I s - A ) has tions relating to multivariable and time-varsring systems no poles, itfollomthatthe poles of thesetransfer have received some attention. However, i n spite of the matrices must be zeros of det ( I s - A ) , i.e., the eigenrapidly expanding literature on the state space apvalues of A . T h a t is not to say that every eigenvalue of proach? a number of issues fundamental to the control A is necessarily a pole; unless further assumptions are problem have been neglected. The object of t h i s paper made, thereis no guarantee that some the eigenvalues of is to fill in some of these gaps. will not be "canceled out." The additional assumptions which are required are that the system should be both PKELIMIN-4KIES AXD N O T L 4 T I O S controllable and observable. That is, is necessary t h a t it Consider the class of linear, time-invariant systems the tzXnnz matrix (B, B , . . . A"-'B), xvhose columns A which can be described b): the pair of vector equations are the columns of the matrices B?A B , through A n P 1 B , should be of rank n , and i t is necessary that the mn X n x=Ax+Bu (1) matrix1 ( C ; A ; . . . CA"-l) n-hose ro\\-s are the rows C y = Cx Du. ( of thematrices C, C A , through CAnP1shouldbe of 4 rank n. T h e role of these assumptions has been discussed Here u and y are the input and output, respectively, a t length elsewhere [ 11- [4]. I t is of some interest to note that is possible t o give it Manuscript received August 10, 1961; revised January 1, 1965. This research was supported in part by the Kational.Aeronautics and an alternative characterization of controllabilitl; in

IKTRODCCTION

1 Thenotation used here is not standard; hoLvever, the use of ( M ; N ) to denote what Lvould ordinarily be written as ( W ,NTjr does make things neater and seems to bejustified.

129

130

IEEE TRANSACTIONS ON AUTOMATIC April CONTROL

terms of the input-to-state transfer matrix. I t is shown Considerthesequence of scalars d, c b , cAn-'b. In in [4] t h a t a necessary and sufficient condition for general, it cannot be assumed that is nonzero, or that d (B, B , . . . An-IB) to be of rank n is that there should cb is nonzero, etc. However, if all the terms in this seA be constant no nonzero row vector N such t h a t quence are zero then clearly u has not effect on y, and, H ( I s - A ) - l B is zero. It also shown may be that hence, the transfer functionis zero. Assuming that this ( C ; C A ; . . . CA"-') is of rank n if and only if there is not the case, define the numberscy and p as exists no constant nonzero column vector H , such that C(Is-AA)-'H is zero. Asimilarobservationhasbeen made by Butman and Sivian [SI. cAi-lb # 0 , and cAjb = 0 (8) Although there are many ways of representing sysforO<j<i--l tems with a given matrix transfer function in state variable form, Kalman [l] has shown that all controllable ifa=O 4-1 = and observable representations of a given matrix trans(9) (,A", if a # 0. fer function are of the same dimension. In the singleinput, single-output case, this is the degree of the deI t will be shown that both and q have a simple intera nominator of the transfer function after common factors pretation in terms of the transfer function. In fact,a is have been canceled. T h e following lemma is an immedithe difference in the degree of the numerator and the ate consequence of this result. denominator polynominals and Lemma 1 q - I = lim saG(s). If y and u are scalars related by (1) and (2), and if S' these equations are controllable and observable, then every eigenvalue of A is a pole of c ( I s - A ) - l b + d , and We will call a the relative order of the system; it occurs the order of each pole equals the multiplicity of the frequently in the sequel. associatedeigenvalue. In terms of this notation it is seen t h a t u is given by Proof: Since the denominatorof the transfer function qy(")-pcAax. Substitutingthisinto ( l ) , thepair of is the determinant of I s - A , i t follows t h a t poles must equations be eigenvalues of A and that no pole can be of higher order than the multiplicity of its associated eigenvalue. If some pole is of lower order than the multiplicity of its associated eigenvalue, then the denominator can be of a t most degree n- 1. This means that the original system has an n - 1 dimensional realization which conare obtained. This pair defines what will be called the tradicts the assumption that the system is controllable inverse system. Since the input to the inverse system is and observable. Y ( ~ ) rather than y itself, this is a slightly different terminology than that used by Zadeh and Desoer [SI. THEINVERSE EQUATIOX Bytakingthetransform of (10) and by using the T h e problem of obtaining a convenient expression for transform of (11) we arrive at the frequency domain the zeros of the transfer functionc ( I s - A ) - l b + d in the equation case where u and y are scalars leads directly to that of deriving a first-order representation for a systemwhose transfer function is the reciprocal the given one. This of systemhasbeenstudiedpreviously [6] inconnection $/,Qare the poles Since the zeros of the transfer function with the controllability properties of dynamic systems of the transfer function z2,/9, and since the latter are and is closely related to the inverse system discussed by eigenvalues of A - bqcAa, we see t h a t the zeros of the Zadeh and Desoer [SI. original system must be eigenvalues of A - bqcA". This By repeated differentiation of (2) one can, bs; using leads a t once to the identity (1) to eliminate x , arrive at the following set of equations (y") =diy/dti). c(I$ - A)-% D

y'0'

+ = cAx + cbu(O)+ dztc')

cx
dZC'0'

4-l det (Is - A

+ bqcAu),'su det (Is - A ) .

(13)

>I(?)

cA2x

+ c f i ~ ( 0+ cbu(') + d ~ ( 2 ) )

. . . . . . . . . . . . . . . . . .
y(n)

C A " .+ cAn-'bu(O) + cAn-?bzC(1) ~

+ . . - cbu(n-1) + dZG(").

(7)

4lthough this expression may appear to be complicated, i t is, in fact, an extremely convenient formula for the transfer function, because it does not require matrix inversion. Since the characteristic polynomial of an n x n matrix isof degree n, this identityplaces in evidence the

I965

Brockett: Poles, Zeros, and Feedback

131

fact that the numerator and denominator polynomials are of order n - cr and n , respectively. From (12) we see t h a t q has the interpretation given above. -4 second important point is that at least one termin the sequence dcb, . . . cAn-'b must be nonzero if the transfer function is to be nonzero. Otherwise stated, is it necessary t h a t ( d , cb, c a b , . . . cA"-'b) shouldbe of rank 1. This condition also arises in the study of output controllability ( 7 j . Thus far it has been shown that a zero must be an eigenvalue of A - bqca". I n order to say more, that in is, order to establish some sort of converse statement, it is necessary to restrict the discussion to controllable and observable systems. In this case the following theorem describes the pole-zeroconfiguration.

Theorem 1

Suppose that the system defined by (1) and (2) is controllable and observable and that ZI and y are scalars so thatthetransferfunction c ( I s - A ) - l b + d is also a scalar. Then the poles of the transfer function are the eigenvalues of A , each pole being of the same order as the muItiplicit>F of thecorrespondingeigenvalue. T h e zeros of thetransferfunctionaretheeigenvalues of A-bqca", the zeros not at the origin being of the same order as the multiplicity of the associated eigenvalue, and the zero at the origin being of o r d e r j - a w h e r e j is the multiplicity of the eigenvalue0. Proof: The statement about the poles is simply a restatement of the lemma. The result relating to the zeros A follows from (13) and the fact that since the original sys- which has a transfer function c(1s - j-'b = (s,+2),:' s ( s + l ) . T h e associated inverse system is temisbothcontrollableandobservable,thedenominator and numerator have no common factors. order In to get a first-order representation for the inverse system we had to "multiply" the transfer function by sa. This explains why the multiplicity of the eigenvalue at the origin is a more than the orderof the zero a t t h e origin. -4 second result on the inverse system which is of interest is given by the following theorem which asserts that the controllability properties of the inverse system A quick check shows that the original system is both are the same as those of the original system. (Keep in controllable and observable but that the inverse system mind t h a t y(") is being regarded as the input to the inis notobservable.The difficulty stems from thefact verse system and not y.) that the original system has a pole a t t h eorigin. If there

rank 12. T o show thatthis implies thatthematrix (qb(A- bqca")qb, . . . ( A - bqca")"-'gb is of rank n, notice t h a t if this latter matrix were not of rank n then there wouldexist anonzerorowvector h such that h(qb, ( A - bqca")qb, . . . ( A - bqca")"-'qb)= 0. By expanding this it may be seen that this can hold only if h(b, ab, . . an-'b)=O. (Recall t h a t p is a scalar and thus can be factored out.) This last statement contradicts the assumption that the original system was controllable and thus establishes the "if" part of the theorem. The converse is more direct; all t h a t is required is to show that if the original system is not controllable, then the inverse system is not either. If the original system is not controllable, then there exists a nonzero row vector h such that h ( b ,a b , . . . An-'b) is zero. This clearly implies t h a t h(qb, ( A - b q c A Q ) q b ,. . . ( A - bpcaa)n-lqb = O and hence that the inverse system is not controllable. I t should be noted that no such result holds for observability. T o see this consider the following counter example. Consider a system of the form

Theorem 2
The inverse system is controllable if and only if the original system is controllable. Proof: I n this proof, and in several of those to follow, some use is made of the fact that an n x m matrix is of rank n if and only if there exists no nonzero t z dimensional row vector H , such that when H i s premultiplied by the matrix the result is zero. An equivalent statement holds for column vectors and postmultiplication. For a discussion of these results see a bookonlinear algebra such as Halmos [8]. Returning now to the proof, assume that the original system is controllable so t h a t ( b , A b . . . A"-'b) is of

are no such poles, i.e., if A is nonsingular, then under the assumptions given the inverse system will be observable, although this will not be proven here.

LINEAR STATE VARIABLE FEEDBACK If all the state variables can be measured, and if the
plant is completely known, then an optimal controller can frequently be realized as an instantaneous function generators whose inputs are the state variables. For this and other reasons i t is of interest to explore the effects of state variable feedback various system properties. on T h e discussior, will be limited to linear time-invariant feedback.

132

IEEE T R A N S A C T I O A r S ON p A iU T O MO L I C A CONTR AT r l
1 1
( c ; ca> = [1

and, thus, the s!-stem is not observable. The transfer function associated with (18) is 2(2~-33)/(~-11)(2~-3); the reason observability was lost is that the addition of feedback moved a pole t o cancel a zero. X = (A - 3K)x 3 u ; y = CX Du. (16) Sotice that the cancellation of a zero by using feedback is not possible if just the output isfed back; this is I t is well known that feedback can alter polelocaclear from the properties the rootlocus. I t is apparent, of tions; i n fact i t will be shown t h a t if the system is con- therefore, that one has considerably more flexibility in trollable, then state variable feedback can be used to fixing the closed-loop poles if the entire setof state variachieve any desired pole configuration consistent with ables can be fed back. Questions which suggest themthe dimension of the s!-stem. T h e following theorem as- selves are: "Can zero locations be moved by the applicaserts that in spite of this linear state variable, feedback tion of state variable feedback?" and, "are there any does not affect the controllabi1it)- of the system. restrictions whatever on the pole configurationwhich can be achieved using state variable feedback?" In the Theorem 3 specialcasewhere u and y arescalarsthefollowing If linear state variable feedback is applied to the sys- theorem resolves these questions. tem (1)-(Z), then the closed-loop system (16) is controlTheorem 4 lable if and onlyif the open-loop system was controllable. Suppose that (1) and (2) are controllable and observProof: If the open-loop systemis controllable then for each initial state x0 and each desired final state x,-, there able and that u and y are scalars. In this case, linear is a n input which drives the s)-stem from x to xi. Let state variable feedback can be used to obtain any de. u' denote this input and let denote the corresponding siredpoleconfigurationwhichisconsistentwiththe x ' dimension of the system. Any number of the open-loop response.Fortheclosed-loopsystems,thesamestate zeros may be cancelled but no new zeros can be introtransfer can be obtained using the input u = u'+kx'. duced. Thus the closed-loop system is controllable if the openProof: I t has been shown [ l ] that by a change of coloop system is. AIoreover, if the open-loop q-stem is not ordinates any single-input, single-output system which controllable then closed-loop the system not can be in the either; any transfer executed b>* the closed-loop system is both controllable and observable can be put can be accomplished by the open-loop system simply by form changing the input to u ' - k x ' . To see that state variable feedback can affect observability consider a single-input, single-output s>-stem of the form

The effect of linear state variable feedback on systems of the form given by (1) and (2) is t o replace u by u - Kx where K is an n x rn matrix of constants, thei j t h element being the gain between the j t h s t a t e variable and the ith input. The closed-loop equations of motion are,therefore,

,]

(19)

3'

11

[;I

.
3' =

This system is both controllable and observable and has atransferfunction of the form (2~-33)/(s-l)(s-2). If state variable feedback is applied through a k of the form (0-0.5) then the equations of motion become

For the closed-loop system we have

Suppose that the characteristic equation associated with the desired pole configuration is s ~ + T , - I s " - ~ . rlsfro. If k is taken to be ( Y O -Po, Y 1 - f i 1 , * * . rn-l-fln-1) then an easy calculation shows that the closed-loop system will have the desiredpole configuration. From this i t is obvious that any or all of the existing zeros can be cancelled. To see that no new zeros can be introduced is more difficult. First it will be shown t h a t

1965

Brockett: Poles, Zeros, and Feedback

133

the relative order of a system is not affected b y feed- of y unless the equationsof the plant areknown exactlv. back, To see this consider the sequence, c b , c ( A - b k ) b I n view of the acute sensitivity problems associated d . . . c ( A - b k ) n - l b which determines the relative order with the generation of the complete set of state variaof the system with feedback.If the relative orderof the bles, it is of some interest to explore the effects of feedopen-loop system is zero or one, then it is clear that the ing back just those state variables which can be genif smallchangesoccurinthe relative order of the closed-loop system is the same; if eratedaccuratelyeven the relative order of the open-loop system is a > 1, then plant, i.e., y and its first a- 1 derivatives. Apparently cAib=O for O < . i < o r - l . Since ( A - b k ) i canbe ex- not a great dealis known about the effects of such feedback on the pole locations. It seems clear that the types panded as Ai-Ai-'bk-bkA'-'+ . . . A(bk) will becon5 ( b k ) i - ' A T ( b k )i, i t is clear t h a t c ( A - b k ) ;b vanishes of configurations which canbeachieved for exactly the same indexes as c A i b doesand t h a t strained but itis difficult to see exactly how. The situation with respect to zero cancellation is quite simple, c(A-bk)"b=cA"b. The inverse system associatedwiththe closed-loop however, and differs from t h a t described by Theorem 4 in t h a t no cancellation can take place. This is an immesystem is, therefore, given by diate consequence of the following theorem which asserts = ( A - b k - bqc(A - bk)")x bgy'"' (21) t h a t if only the first or-1 derivatives are fed back, the closed-loop system is both controllable and observable. u = - pc(A - bk)% qIJ("). (22)

Fromourpreviousremarksitis seen thatthe ferential equation can be simplified to

X

dif-

Theorem 5'
Suppose that (1) and (2) are controllable and observable and that 21 and y are scalars. If a loop is closed about this system byfeeding back a linear combination of y and its first a - 1 derivatives, then the closed-loop system will be controllable and observable. Proof: From (7) i t is clear that y and its first a - 1 derivatives can be generated from the state vector x, and, thus, the controllabi1it~- the closed-loop system of is an immediate consequence of Theorem 3. Suppose the that feedback signal is aoy+aly(l) .. so that the closed-loop equations of motionare

( A - bqcila)x

+ bpy'"'

(23)

and,thus,theeigenvalues of the closed-loopinverse equation are the same as those the open-loop system of and no new zeros can be introduced. I t should be pointed out that niorgan [9]-[lo] has obtained a result similar to Theorem 4 using somewhat different methods.

DERIVATIVE FEEDBACK

+ -

Inmostsituationsitisimpossibletomeasurethe state variables directly; instead they must be calculated = Fx bzt; from the terminal variables u and y . If the system has no zeros then y and its first n - 1 derivatives constitute F = A - aabc - albcA - . . al;-lbCA"-l (24 an admissible set of state variables, and even though it may be difficult to measure these accurately practice, in y = cx dl.1. (25) i t is clear that the derivatives actually exist provided that is at least bounded and piecewise continuous. If this system were not observable, then there would What is more important, even if the pole locations shift, exist a nonzero column vector N s u c h t h a t ( c ; c F ;. . the indicated derivatives still exist, and y and its first cFn-l)h= 0. Since c A i b is zero for 0 < i < k , i t follows n- 1 derivatives summarize the state of the system. that this implies t h a t ch = c A h = ' cA"-lh = 0, and, Contrast 'this with the case where the system has hence, i t would follow that the original system was not zeros, t h a t is, the case where a#%- 1. From (7) we see controllable. This contradiction establishes the theorem. t h a t in this case y ( i ) will not necessarily exist if ;>or A CANONICAL FORX OPTIMAL FOR CONTROLLERS unless some differentiability conditions are imposed on u. In this case exactly elements of the state vector can Recentlytherehasbeenconsiderableinterest a in a be obtained fromy and its first a - 1 derivatives, the re- variant of the time optimalproblem in which the object maindermustbegeneratedbydifferentiatinglinear is to take the output to some point, say zero, and to combinations of u,y, and the derivatives of ZL and y . keep it there for some period of time, or for all future Unfortunately, if changesoccur in the pole-zerocontime. This differs from the usual problem in t h a t it is figuration, the linear combinations variables that one not necessary to hit a specific point in the state space of is attempting to measure may no longer be state varibut instead one may hit any point in state space for ables, and, hence, their derivatives will not necessarily which y is zero, provided only t h a t i t is possible to keep exist. Thus, large errors may be introduced if one at- y a t zero for the desired period of time. A little thought tempts to feed back more than the first - 1derivatives is all t h a t is necessary t o convince the reader thatif the a

--

134

IEEE T RA N S A C TIOLYS OLV A U T O M A T I C C O N T R O L

April

system is linear and constant and if ZL is amplitude limited, then this problem is equivalent to the standard time optimal problem prozlided the system has n o zeros. A discussion of this can be found in [11]-[13]. If zeros are present the solution this problem is sigof nificantly different in that the value of the input z1 for t larger than the response time t* is not zero and can not be generated by an ideal relay. I t can, hon-ever, be generated as a homogeneous solution of the inverse equation. This last point is fairly clear if the inverse s>-stem (10) is examined with t h i s problem i n mind. In fact, if
_-

and only if the system was output-controllable, i.e., if ( c b , c A b , . . . cAn-'b) was of rank 1. In the multivariable case the situation is more complicated. In the case D=O the necessar>- and sufficient conditionsforthe transfermatrix t o benonsingular(equivalenttothe conditions under which the inverse s\.stem exists) have been given previously [4]. trivial modification of the LA derivationgiventhereestablishesthatingeneralthe necessarJ- and sufficient conditions for the inverse equation to exist is that the m a + m by 2nrn+vz dimensional matrix

. . . . . . . . . . . . . . . . . . . . . . .
10 0 0

...D

CB

. . . CAn-'B

should be of rank mn+nz. ,Assuming that this condition is fulfilled, i t will be possible to calculate the inverse system for any specific system although giving a general formula seems to be = ( A - bqcAa)x (26) impossible. T h e difficulty here stems from the fact that in the multivariable case, solving (7) for zz is compliu = - qcAax. (27) cated by the fact that the first matrix in the sequence Thus,after the end point reached the optimum input D, CB, CAB, . . . CAn-'B whichisnonzero is will not can be generated bs; the inverse equation. Before this necessarily be nonsingular. If it is not, then the number time, of course, the input can be generated by an ideal cy loses much of its significance unless onegives it a relay driven by the adjoint system. The general optimalmatrix interpretation. controller,shown i n Fig.1,therefore,consists In the multivariable case the eigenvalues of the of the inadjointsystem in thefeedback loop plzrs theinverse verse equation do not correspond to the of particuzeros the>- correspondto the system operating in a feedforward mode to generate the lar transfer functions. but rather values of u ( t ) for t > t * . *Additional material and ex- zeros of the determinant of the transfer matrix. It has amples may be found i n [6]. been shown that these zeros play a fundamental role in least-squaresoptimizationtheory[14],andthatthey are importantin determiningif a plant can be decoupled INVERSE SYSTEM by state variable feedbackl j ] .liotice, however, t h a t it [ will be impossible to decouple a multivariable plant by any means unless the matrix M defined by (28) is of rank mn+m and that this is also a necessary condition formostmultivariabletrackingproblems to havea ADJOINT solution. SYSTEM

we require y t o be identically zero theny ( " ) must vanish also, and, hence, we see t h a t for some choice of x(t*) the optimum value of ~ ( f ) for t>t* is given by

COSCLVSIOXS

Fig. 1. -4 canonical form for optimal controllers.

~'IULTIVARI.L\BLESYSTEMS

-Although some of the results and methods of the previous sections carry over to multi-input multi-output systemswithlittlemorethanachange of notation, othersdonot. I t seemsworthwile toindicate briefly here whatthe difficultiesseem to be. Onlythecase where the number of inputs equal the number outputs of will be considered here. T h e first question which arises relatesto theexistence of the inverse equation. In the single-input, single-output case i t was shown that the inverse system existed if

The object of this paper has been to consider several problems related to the relationship between state variable and transfer function representations of systems and to examine the effects of feedback on certain system properties. X convenient expression has been given for the zeros of a transfer function and the effects of state variable feedback on controllability, observability, and the pole-zero configuration has been examined. A n inverse s!-stem has been defined and some of its properties were examined; a possible application in the implementation of optimal control laws was indicated. In the final section the extension of these resultsto the multivariable case was briefly examined.

1965

IEEE T R A N S A C T I 0 : Y S OLY A U T O M O T I C CONTRA L

ACKNOWLEDGMENT

135

The author would like to thank Prof. AI. D. Alesarovic of Case Institute of Technology and Prof.31. -4thans of the AIassachusetts Institute of Technology for their helpfulsuggestions. He would alsolike to thank Dr. B. S. AIorgan, Jr. of the Xir Force Offce of Scientific Research for providing him with a cop). of his thesis.

REFERENCES
[ l ] Kalman, R.E., The mathematical description of linear d5mamical systems, SIa41fJ . 012 Control, vol 1, 1963, pp 152-192. [2] Gilbert, E. G., Controllability and obsenabilitl-in multivariable control systems. S I ; 2 X J . on Control. vol 1. 1963. DD 128-151. [3]Zadeh, L . -4. andC. .A. Desoer, Lil-zear System i l ; e o v ~ , . S e w York: lIcGra\\--Hill, 1963. [4] Brockett, K. \V. and \I. D. llesarovic, The reproducibility of multivariable sl-stems, PTeprinls Joint 4utomatic Control Conference. Stanford, Calif.,. Tun 1964 (to appear in Xath. A n a l y s i s _.
[j] Butman,

Applications). S., and R. Swan, On cancellations, controllabi1it:- and observabilitv. IEEE Trans. o n Automatic Control. 1-01 XC-9. Jul 1964, p p 3177318. 161 Brockett, R. I\., Thein\-ertability of dynamicsystemswith
aud

application control, to Ph.D. dissertation, Case Institute of Technology, Cleveland, Ohio, 1963. [7] Kreindler, E., and P. E. Sarachick, On the concepts of controllabilityandobsen-ability of linearsystems, I E E E Trans. O R Azrtonmtic Control, vol .IC-9, Jan 1964, pp 129-136. [8] Halmos, P. R.,F i d e Di?nensional Vecto? Spaces, Princeton, X. J.: Van Sostrand, 1958. [9] Morgan, B. S., The synthesis of linear multivariable systems by state variable feedback, dissertation, Ph.D. University of Michigan, .Ann .Arbor, 1963. [lo] klorgan, B. S., The spthesis of single variable sg-stems by state variablefeedback, Proc. Allerton Conj. on Ciycnit a n d System Theory. I-niversity of Illinois, L-rbana. Xov 1963. [ l l ] Lee, E. B., On the time-optimal controlof plants with numerator d g a m i c s , I R E Trans. o x dnttowzatic Coxtrod, vol AC-6, Sep 1961, pp 351-352. [12] Athanassiades, h.1.: and P. Falb, Time optimal control for plants with numeratordynamics, I R E T r a m . on Automatic Control, VOI -IC-7, J u I 1962, pp 4i-i;O. [13] Harveg-, C. Determinmg switching the criterion timefor optimal control. J . o j Xuth A n a l y s i s a n d .4pplications, vol 5, 1962, pp 2.5-268. [14] Brockett, I<. \X-., and XI. D. Nesarovic, S!-nthesis of linear multix-ariable systems. -4 I E E Trans. (Applications in I t z d z ~ s t ~ y j , yo1 81, Sep 1962, pp 216-221. [ l j ] klprgan,B. S., The sl-nthesis of linearmultil-ariablesystems usmg state variable feedback, Preprints Joint dzdowzatic Control Crmference, Stanford, Calif., Jun 1961.

Dynamic Programming with Reduced

Computational Requirements
ROBERT E. LL4RSOAAbstract-The purpose of this paper is to describe in detail a computational procedure for solving optimal control problems called stateincrement dynamicprogramming. Thisprocedurehas computational requirements which are considerably less than those of conventional dynamic programming, but it still retains the general o applicability and other desirable featuresf the conventional method. In one example the reduction in high-speed memory requirement is fromlo6toabout 100 locations. Similarreductions incomputing time can also be obtained in some cases. The new procedure, like the conventional method, is based on iterative application of Bellmans principle of optimality. It differs from the conventional method in the choice of the time intervalover which a given control is applied. Instead of using a fixed interval, the new procedure determines the time interval as the minimum time required for at least one the state variables change by one increof to ment. As a resultof this choice of interval, thenext state for any given control is known to lie within a small neighborhood of the point at which control is applied. By using this result it ispossible to compute optimal control in units, called blocks,! that cover a relatively long timeintervalbutasmalldistance alongeach state variable. By using only one or two high-speed memory locations per state in the block-an almost arbitrarily small numberof locations-it is possible to compute optimal control throughout the block. Specialized computations near the boundaries of the block permit the optimal trajectories any desired degree of freedom in passing from block to block. Manuscript received July 20, 1964; revised December 28,1964. The author is with Stanford Research Institute. P;Ienlo Park, Calif. This technique has been applied in a computer program which calculates minimum fuel trajectories for supersonic (Mach 3) aircraft under a variety of conditions and constraints. The basic program can be used both for a detailed evaluation of an aircraft design and for real-time control of an aircraft.

I. INTRODUCTION N E O F T H E 3IOST important techniques developed in recent years for the solution of optimal control problems is Bellmans dynamic programming [1]-[3]. This technique solves, a t least in principle, an extremely large number of important optimization problems. However, because the extremely of largecomputationalrequirements, called by Bellman thecurse of dimensionality [ 2 ] , [SI, onlyrelatively existing comsimple problems have been solved an puters. T h e computational procedure of state increment dynamic programming, which is described in this paper, provides a method for greatly reducing - the computational requirements from thoseof conventional dynamic programming, while atthesametimeretainingthe many desirable properties of the conventional method. T h e decrease in high-speed memory requirement is alwaysconsiderable,usuallyseveralorders of magni-