1488

AIAA JOURNAL

VOL. 6, NO. 8

Optimal Programming Problems with a Bounded State Space

JASON L. SPEYER* AND ARTHUR E. BRYSON jR.f Harvard University, Cambridge, Mass.
A new set of necessary conditions are presented for solutions to optimal programming problems with a state variable inequality constraint (SVIC). On a constrained arc, the dimension of the state space is reduced, and the influence functions associated with this reduced state space are shown to be unique and continuous across junctures with unconstrained arcs. It is also shown that unconstrained arcs must satisfy certain "tangency" constraints at both ends of a constrained arc and that explicit use of this fact must be made if the SVIC is adjoined directly to the performance index.

Downloaded by Beihang University (CNPIEC - XI'AN BRANCH) on May 12, 2013 | http://arc.aiaa.org | DOI: 10.2514/3.4793

1.

Introduction

and on a constrained arc let c(x,u}t) be defined as the scalar c(x,u,t) = 8&(x,u,t) = 0 (4) Now y must be identically zero on a constrained arc, so the dimension of the state space is reduced to n-p. Let us choose n-p functions ~zi(x,t)

^EVERAL investigators have discussed necessary condi^ tions for solutions to optimal programming problems with a state variable inequality constraint.1"5 Most of these investigators recognized that, at a point where the system enters onto a constrained arc (an "entry point"), all feasible unconstrained arcs passing through this point must satisfy certain tangency constraints, namely, these arcs must have zero values of the state variable constraint function and all of its time derivatives that do not involve the control variable (say p-1 of them). Since control of the constraint function is realized only by changing its pth time derivative, no finite control will keep the system on the constraint boundary if the path reaching the constraint boundary does not meet these tangency constraints. In this paper, we point out that the tangency constraints also apply to the unconstrained arc at a point where the system leaves a constrained arc (an "exit point"). These exitpoint tangency constraints are satisfied automatically if the necessary conditions of Ref. 3 are used. However, if one uses the "direct adjoining" approach of Ref. 4, we shall show that explicit use must be made of the tangency constraints at both ends. 2. Reduced Dimension of the State Space on a Constrained Arc

(5)

so that the p-vector y and the n-p vector z together span the n-dimensional x-space. In particular, the n by n Jacobian matrix

must be nonsingular in the bounded state space. The implicit relation (4) defines u as a function of x and t. On a constrained arc (y = 0), it defines u as a function of z and t. Thus on a constrained arc it is possible to express the system equations in the form z = g(z,t) 3. Problem Statement
(6)

Dreyfus7 made use of the fact that the dimension of the state space is reduced on a constrained arc. If the SVIC is s(x,t) < 0 and the system equations are x = f(x,u,t) x} an n-vector; f, an ^-vector function; u, a scalar (2) s, a scalar function (1)

Find u(t) mt0 < t < tf to minimize J = 4>[x(tf),tf} subject to the constraints x = f[x(t),u(t\t] *[x(tf),tf] x(k) and fo specified = 0 (8) (9) (10) + f" L[x(t),u(t),t]dt
J to

(7)

then, as in Ref. 3, we shall call (1) a pth-order SVIC if p times derivatives of s(x,t)} substituting f(x,u,t) for x from (2), are required before u appears explicitly in the result. Let y(x,t) be defined as the p vector

8[x(t),t] < 0

where < and s are known scalar functions, x is an n vector and \fr" is a q vector (q < n + 1). 4. Necessary Conditions on Unconstrained Arcs

where sw =

drs dir

(3)

Presented at the AIAA 1968 Joint Automatic Control Conference, Ann Arbor, Mich., June 26-28, 1968 (no paper number;

Assume a constrained arc for t\ < t < tz and unconstrained arcs for fo < t < t\ and ti<t<tf. Then on the unconstrained arcs, solutions must satisfy Eqs. (8) and where H = \Tf + L

published in bound volume of conference papers); submitted November 28, 1967; revision received April 3, 1968. * Graduate Student, sponsored by the Raytheon Fellowship
Program, Division of Engineering and Applied Physics. t Professor of Mechanical Engineering, Division of Engineering and Applied Physics. Associate Fellow AIAA.

(H) (12)

0 =

The influence function or Lagrange multipler X is an n vector and H is the variational Hamiltonian. At the terminal point,

AUGUST 1968

PROGRAMMING PROBLEMS WITH A BOUNDED STATE SPACE

1489

the solution must satisfy (9) and

(13)
where < =

For a stationary solution, the coefficient of dz(ti) and dt must vanish, ^T = \TN (27) at t = t

0 = H + \T(Myt + Nzt)

aU = *i

(28)

H(tf) = -

(14)

where the Lagrange multiglier v is a q vector of constants. At the initial point, the solution must satisfy the boundary conditions of (8).

5.

Necessary Conditions on a Constrained Arc

On a constrained arc,

y(x,t) = 0
Downloaded by Beihang University (CNPIEC - XI'AN BRANCH) on May 12, 2013 | http://arc.aiaa.org | DOI: 10.2514/3.4793

(15)

where the p-vector y(x,t) was defined in (3), z g(z,t)f and where the n-p vector z was defined in Eqs. (4-6). In addition, a reduced set of influence functions /z, of dimension n-p; is defined by

Oz

where G = ^ g(z^

(^)

(16)

where L(z,t) = L(x)u)t)1 u is eliminated through (4), and x is expressed in terms of z with y = 0 through (3) and (5).
6. Necessary Conditions at a Juncture between a Constrained and an Unconstrained Arc

For the unconstrained arc in to < t < ti, the n specified inital state variables of (8), the p conditions (17), the n p conditions (27), and the single condition (28) provide 2n + 1 conditions for the 2nth-order differential equation system of Eqs. (8) and (11) with the unspecified time t\. Similarly, for the unconstrained arc in Z2 < t < /, the p conditions (17) and the n p + 1 conditions (27) and (28) applied at t = t% and the q + n + 1 conditions (9, 13, and 14) provide 2n + 2 + q conditions for the 2nth-order system (8) and (11) with the unspecified times t% and tf and the q constants v. If y and z were used as the state variables on the unconstrained arcs as well as on the constrained arc, the multipliers associated with z are seen from (28) to be continuous across the juncture points. The example to follow illustrates this point. In general, these two-point boundary-value problems for the two unconstrained arcs cannot be solved separately since they are coupled through z(t) and id(t). However, there are cases in which they can be solved separately (see Ref. 8).
7. Comparison with Previous Work

At a juncture between a constrained and an unconstrained arc, it is necessary that the unconstrained arc satisfy the tangency constraints (see Sec. 1):

[x(td,ti] = 0

i = 1,2

(17)

The quantities /j() may be expressed in terms of X() as follows. We may write the augmented performance index as

(H - \Tx)dt +
7 i +

P (G - ^z)dt + J ftf (H - \T)dt h^

(18)

where the values of the variables just before and just after the juncture are indicated by ti~ and ti+, respectively. From (18), it follows that

dJ = -\T 5x\t^- -VT 54+fa" + (H - \TX)\tl- dk ~ (G - v?z)\h*Mi + However

(19)

=
dz
or

f y* ~

... dx + ... \ d t L z* _ _ ' z* J

~ yt ~]

(20)

dx = M(dy - ytdt) + N(dz - ztdt)

(21)

In Refs. 3 and 7, the SVIC was regarded as an entry-point tangency constraint [Eq. (18) with t = ti]y followed by a control-and-state variable constraint [Eq. (4)]. Any solution satisfying these two constraints automatically satisfies the exit-point tangency constraint. The transversality condition (28) at the exit point was stated in Refs. 3 and 6 in a way such that continuity of u across the exit point is assumed. Although this will often be the case, it is not necessary. Use was not made of the reduced state space on constrained arcs in Refs. 3 and 6; as a result, the influence functions X(0, i = 1, . . . n, are not uniquely determined on the constrained arc. However, it is clear from the present work that n-p linear combinations of the X/s are uniquely determined [see Eq. (27)] by the conditions stated in Ref. 3 once a suitable choice of n-p state variables is made as in (5). In Ref. 4, the necessity of tangency constraints on the unconstrained arcs was not recognized. Hence, there were too few necessary conditions stated to determine an extremal solution satisfying the constraints. It is shown in the Appendix that explicit use of the tangency constraints is required at both the entry and the exit points when the inequality constraint (1) is adjoined directly to the performance index as was the case in Ref. 4. When this is done, the influence functions \i(t), i = 1, .. ., n are uniquely determined on a constrained arc and are either continuous or have unique discontinuities at the entry and exit points. In the Appendix, the necessary conditions at the entry and exit points are similar, just as they are in the previous section [see Eqs. (28) and (29)].

where
8. Analytic Example

[M\N] =
and where ( )x and ( )z means b( )/d# and d( the juncture t = fr, we have

(22)

Now, at (23) (24) (25)

dyfa) = 0
) + (*i~)<fti Substituting (22-26) into (20), we have

The preceding discussion is illustrated here by a problem whose three-dimensional state space is bounded by a secondorder SVIC. The optimal unconstrained arcs for this problem cannot be calculated separately and then joined together as in Ref. 8. The problem is to find the control program u, over the interval te [0, 3 ] which minimizes
1

?dt

(29)

subject to the constraints

dJ = [p? - \TN]t=tidz(ti) + [^ (Myt + Nzt) +

H - G]i=hdtl+ ..
(26)

h(G) = 1.6, h(S) = 0 v(0) = -1,0(3) = -1

(30) (31)

1490

J. L. SPEYER AND A. E. BRYSON JR.

AIAA JOURNAL

X. - -X

(42)

X a = -X,

(43)

On the constrained arc, the Hamiltonian is

G = fja)e(t) + 32
with the associated multiplier equation
A = 0

(44)

(45)

At the junctures to the constrained arc, Eqs. (28) and (29) must be satisfied where

o on
M =
Downloaded by Beihang University (CNPIEC - XI'AN BRANCH) on May 12, 2013 | http://arc.aiaa.org | DOI: 10.2514/3.4793

rn
N =

r-cn
zt = 0,yt =

Lo ij

10

LoJ

L-W.J

(46)

then
(47)

KHJNOARY

1 1

2 I

X v (t)

H
Fig. 1 Time history of the state variable.

CONSTRAINT I ____ BOUNDARY P

a(0) = 0, a(3) = 0 a = u with a SVIC given as

(32)

(33)

s = v - vc() > 0
where vc(t) is given as

(34)

vc = -W + I2t - 8 (35) The variables h, v, a, and t can be thought of as position, velocity, acceleration, and time, respectively. These state variables on the constrained arc are subscripted by c. Consider the new state space as denned in Sec. 2,

\v - vc(tr

(36)

\_a oc(0_ 2 = h c =u+ 8 h = ve(t)

(37) (38)
(39)

On the constraint boundary, the system equation is only

The variational Hamiltonian is defined on the unconstrained arcs as

H = \hv + \va + \au + u2/2 \H = 0

(40) (41)
Fig. 2 Time history of the influence functions and control variable.

The corresponding Euler-Langrange necessary conditions are

AUGUST 1968

PROGRAMMING PROBLEMS WITH A BOUNDED STATE SPACE

1491

\hVe(t) = \hV + \v(a - ae) + X a (^ - uc)

(48)

At the junctures, tangency requires v = vc and a = ac, so Eq. (48) implies that at the juncture

(49) In general, it can be shown that if the Hamiltonian is regular the control will be continuous across a juncture point.9 The time histories of the state variables and influence functions which satisfy the previous necessary conditions are given in Figs. 1 and 2, respectively. The dashed paths are optimal solutions without the inclusion of the SVIC. The solid paths are optimal solutions that include the SVIC of (34). The constrained arc lies between the juncture times 1 and 2. The reduced state space is represented only by A, and the corresponding multiplier is X&. The values of \v and X a on the constraint boundary are of no consequence. The optimal unconstrained arcs cannot be calculated separately because of the need to satisfy the initial and terminal position constraints.
Appendix: Handling State Variable Inequality Constraints by Direct Adjoining

The X vector is, in general, discontinuous at both the entry and exit points of the constraint boundary, but these discontinuities are unique. These discontinuities can be expressed in terms of x and X on the unconstrained side of the juncture to the constraint boundary. Let t* be the time of a juncture between constrained and unconstrained arcs (either an entry or an exit point). Then by adjoining the tangency conditions at the juncture points, the necessary conditions corresponding to (28) and (29) are
u

Downloaded by Beihang University (CNPIEC - XI'AN BRANCH) on May 12, 2013 | http://arc.aiaa.org | DOI: 10.2514/3.4793

(A8) (A9)

where ( ) c denotes ( ) on the constrained side of the juncture, and no subscript denotes ( ) on the unconstrained side of the juncture, vi is a p vector of multipliers associated with the tangency conditions (18). The p-equations (A5), the (n + 1) equations (A8), and (A9) determine the p-quantities vf, the n-quantities Xc(^-), and the time ti in terms of X() and the state (since tyfox is a function of the state) on the unconstrained side of the juncture. It is then not surprising that if (A8) is substituted into (28), JJL is found as a linear combination of Xc independent of the ^/s. Thus in the lower-dimensional state space, the necessary conditions obtained here reduce to those of the main text.

In this scheme the SVIC (10) is directly adjoined to the performance index (7) by a Lag-range multiplier 77 in the manner of Chang.4 However, it is also necessary to adjoin to J the tangency conditions at the juncture points to the constraint boundary. The variational Hamiltonian is now defined as

References
Gamkrelidze, R. V., "Optimal Processes with Restricted Phase Coordinates," Izvestia Akademiya Nauk SSSR, Ser. Mat. 24, 1960, pp. 315-356; also Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., and Mischenko, E. F., The Mathematical Theory of Optimal Processes, Interscience Publishers, New York, 1962, English transl., Chap. VI. 2 Berkovitz, L. D., "On Control Problems with Bounded State Variables," Journal of Mathematics and Analytical Applications,
1

H = L+ \Tf +

(17 = 0, s < 0

( n y o, s - o

(Al)

The Euler-Lagrange equations are then

Vol. 5, 1962, pp. 488-501. 3 Bryson, A. E., Jr., Denham, W. F., and Dreyfus, S. E., "Opti-

(A2)

dw

dw

dw

(A3)

On constrained arcs, we may reduce the dimension of the state vector to n p by using the p equalities

s(x,t) =

,t) = ... = s("-(x,t) = 0

(A4)

In a similar manner, we may reduce the dimension of the X vector to n p by using the p equalities

Hu(x,\,t) = Hu^(x,\t) = ... = Hj*-(x,\t)

In (A 5) , u was expressed in terms of x and t by

= 0 (A5)

s<V(x,u,t) = 0
and 77 is determined in terms of x, X, and t by
HvW(x,\,t,ri) = 0

(A6)

(A7)

mal Programming Problems with Inequality Constraints I: Necessary Conditions for Extremal Solutions," AIAA Journal, Vol. 1, No. 11, Nov. 1963, pp. 2544-2551. 4 Chang, S. S. L., "Optimal Control in Bounded Phase Space," Automatica, Vol. I, 1963, pp. 55-67; also TR 400-37, Aug. 1961, Dept. of Electrical Engineering, New York Univ. 5 Dreyfus, S. E., Dynamic Programming and the Calculus of Variations, Academic, New York and London, 1965. 6 Denham, W. F. and Bryson, A. E., Jr., "Optimal Programming Problems with Inequality Constraints II: Solution by Steepest-Ascent," AIAA Journal, Vol. 2, No. 1, Jan. 1964, pp. 25-34. 7 Dreyfus, S. E., "Variational Problems with State Variable Inequality Constraints," Ph.D. dissertation, 1962, Harvard Univ.; also Paper p-2605, July 1962, Rand Corp. 8 Speyer, J. L., Mehra, R. K., and Bryson, A. E., Jr., "The Separate Computation of Arcs for Optimal Flight Paths with State Variable Inequality Constraints," Proceedings of the Colloquium on Advanced Problems and Methods for Space Flight Optimization, Liege, Belgium (held June 19-23, 1967), Pergamon Press, New York. 9 Paiewon&ky, B. et al., "On Optimal Control with Bounded State Variables" Rept. 60, July 1964, Aeronautical Research Associates of Princeton.