DISCRETE MAXIMUM PRINCIPLE N. N. Dzyuba and B. N.

Pshenichnyi
UDC 519.8

In this paper we consider the question o f necessary conditions for an extremum for the following problem. Let X be a locally convex linear separated topological space. We are given a point-set mapping a(-) which puts each point x E X into correspondence with a set a(x) in this space. It is required that the function f(x~ ) be minimized under the condition that
xk+~Ea(x~),

k=0,1 ..... m--l,

(1)

and x 0 is fixed. Below it will be shown that a necessary condition for a minimum can be written in the form o f a maximum princ.iple which generalizes the known one [ 1]. For the case of a convex mapping this problem has been considered in [2]. It should, however, be mentioned that in the case o f a sufficiently general mapping a(.) it is impossible to formulate such conditions in a way that would not lead to stringent hypotheses in particular cases. Therefore, the general result given below should be considered as a scheme for obtaining conditions for a minimum in particular cases. In this connection, the basic theorem on a maximum principle has been formulated with hypotheses which require special verification in each concrete case in order to obtain the most precise result. However, this verification can be carried out in a relatively simple way on the basis o f well-known results in

[1, 3, 4].
We remark that the problem considered in this paper has been studied under other hypotheses in [5]. 1. Let X and Y be locally convex separated linear topological spaces, and let Z be their direct product, i.e., Z = X X Y. We denote by X* and Y* their topological dual spaces, i.e., the spaces o f continuous linear functionals. We will denote the value o f the functional x* E X* (respectively, y* E Y*) on the element x(y) by (x*, x) (respectively, (y*, y)). It is clear that Z* = X* X Y* and ( z * . z ) - = ( ( x * , x ) + ( t j * , y ) ) , if (x, y) is the pair defining z and (x*, y*) is the pair defining z*. If a(-) is a many-valued mapping which places the point x E X into correspondence with the set a(x) C Y, then it determines a set graf a = {(x, y):y E a(x)}, which lies in Z. On the other hand, if the set graf a C Z is given, then it uniquely determines a mapping
a (x) = {y : (x, y) E graf a}.

Suppose now that z ~ graf a. We will say that a cone K z C Z defines a cone o f admissible directions if for all p E Kz, p = (Pl, P2), Pl E X, P2 E Y, there exists a function a(X) E Z, X ~> 0, such that a(X)X -1 ~ 0 for X -~ + 0, and
z -~ Lp q- ~ (k) E graf a

(2)

for sufficiently small X. In other words, the condition y +kp2 q - % ( k ) E a (x + Lp~ q - ~ (k)) must be fulfilled for sufficiently small X, where x and y are the components of z and a~ (X), c~2(X) are the components of c~(X). We denote by K~* the cone dual to Kz, i.e.,

K ; = tz*EZ" : ( z * , z ) > O, v z E K ~ } 9
We define the dual cone in an analogous Way in all the remaining cases appearing in the following exposition: Definition. The point-set mapping

a~ ~') = {x" : (-- x*, V) E K;}

Translated from Kibemetika, No. 2, pp. 46-49, March-April, 1975. Original article submitted May 15, 1974.
019 76 Plenum Publishing Corporation, 22 7 West 17th Street, New York, N. Y. 10011. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in atLv form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission o f the publisher. A cop), of this article is available from the publisher for $15.00.

226

is said to be dual to the mapping a(.) at the point z -- (x, y). Generally speaking, the given definition determines the dual mapping in a nonunique way. (2). We denote by It is

K~ ~ the cone of all directions p such that for some function a(?,) = o(k) we have for small X the condition
max Any cone K ~~ - : K~ will be a cone o f admissible directions, and it determines some dual mapping. /

moreover clear that K; ~_(K~-") *, and, therefore, corresponding to KF "x

a;(g') ~ (am~)*(g *) , where ( a* a~ ,

is the dual mapping

We could define the dual mapping as the mapping corresponding to KT ~ . the cone K~~ as a rule it happens that, although the cone K~ does not coincide with K~
IVI~IX

However, in concrete problems Moreover,

Kz

cannot always be determined, while some cone K~ can be determined relatively easily. , the closure o f

coincides with

K m"~ which implies the equality K* = (K~max)*, wlfich means that we have the coincidence a* = (am~) * Therefore, we retain in the following some indeterminacy in the definition o f a~*, the more so, as it will not !cad to any complication o f the exposition. Remark. The given definition is entirely consistent with the definition given in [2] for the convex case, as is clear from Lemma 2 o f [2]. Indeed, it is clear from this lemma that for those y* for which the dual mapping in [2] is defined it coincides with that given above. We will give examples o f dual mappings for some special cases which wi!l be o f interest to us in the following: Example 1. Let X = E n, Y = E n, a ( x ) = {y.~- l ( x , u ) : u E U } , where U ~ E ' , f : E " a convex set.

X E'-.+E" , and f(x, u) is a continuously differentiable function o f its arguments, while U is

Then

Let z = (x, y), where y = f(x, u o), u o E U.

K. = { 0~,. p~) : p~ = f'~(x, u o) p~ + f. (x, u o) v, vEV~, },

where V.,,--- {r~: v ~- ?(u --u0), u E U, 7 ~ O}, and f~ and fd are the partial derivatives o f the mapping f with respect to x and y. Indeed, y + ~,p~+ a~ (;~) --- f (x + ~p~ + ~ (~,), u o + ;w), where a~ (X) -= 0, % (k) = / (x + k/)~, u 0 + 7,v) - - f (r, u0) - - ~ if; (x, '~0)P~ + f~ (x, u 0) v), and it is easily verified that ~2 ().))'-~-+ 0 , while u o E 3.v E U for small X. It is easy to verify that K- = {(x*,g') :x" ----- - (/;(x, %))'y', (y',f~(x, ~a)) ~ O, u E U}, where (f~)* is the matrix conjugate to f~. if Therefore,

a2 (.q') = {x* : x* = (f;(x, u0))*g*}, (y',f',(x, u o ) ( u - - % ) ) :a 0,

and a~ ( g ' ) = 2~ in the opposite case. uEU,

(3) (4)

Thus, in the given example the vector x* is uniquely determined with respect to y* by the mapping az*(y*). Example 2. Let X = E ~, Y = E "~, % (x, g), i = 1. . . . . k, be continuously differentiable functions of their arguments, whose gradients with respect to x and y will be denoted by ~o~ y) and 9~y(x, y), respectively. Let ~o(x, y) E E k be a vector with components ~oi(x, y). We define a mapping a by means o f its graph graf a = ((x, y):~o(x, y) ~< 0}. We make the following hypothesis: if z ~ graf a and

227

l(z) = {i:%(x,y)

= O,

i = ! .....

le},

then the vectors ~ y ( x , y), i E I(z) are linearly independent.

Under these conditions it is possible to show that

K. = {(p,, p~) : ~ ( x , y) p, + % (x, y) p~ ~< 0, i E I (z)}.

Then by the F a r k a s - M i n k o w s k i lemma ( [ 6 ] , p. 218) I(*~ = {(x*, V ) . x = ( ~ ) ~ (x, V)) v, y = (~'~(~)v(x, ~))"v, v ~ O, v E Ez(~)},

where ~o~(,) x and ~Ol(z)y are matrices whose rows are ~O.;x and 9;y, i E I(z), respectively. F r o m this it follows that

a; Cu') = i - - (q0j~,)+(x,

v))" v : v + = (~)(~)~ (x, y))+ v, v >!

0}.

(5)

Thus, a~*(y*) is n o n e m p t y only if the equation y* = ~o~(z)y(X, y)v has a positive solution v. It is convenient to rewrite the formula (5) in the following form, in order to make it clear that it does n o t have anything to do with the set I(z):
a~ (V') -- {-- (% (x, y)) v : y* = (% (x, y))
9 * 9 *[j

, o >I 0, (v, ~ (x, y)) = 0},

(6)

where now v

Ek,

| and ~o x and ~oy are the partial derivatives o f the mapping ~o(x, y).

2. We return now to the original problem. Thus, suppose we are given a multivalued mapping a which maps x E X into some subset a(x) o f the space X. It is required to minimize the function f ( x m ), x m E X, under the condition that xk+16 a (x~), k = 0 . . . . . m - - I, and x o is fixed. This problem can be considered as an optimal control problem in discrete time. We will suppose that the function f ( x ) is differentiable at each point in the sense that there exists at each point x o E X a linear functional f~(x o) E X, such that l im [ (x~ q- ~'p + c, (~,)) - - [ (Xo) = ([~ (x0)' P) for any p E X and for any function a(X) E X which satisfies the uniqueness condition a ( ? , ) X - ~ ~ 0 for X $ 0. Any set of points x~, ..., X~n satisfying (7) will be called a trajectory and will be denoted by x . In the general case the ~ p e r space X m. wave symbol will in the sequel mean that we have to do with some element of the We consider in X m the sets
k = O..... ITt - l },

(7)

Let x ~ be an optimal trajectory.

M =

{ 2 ; x k + l E Cl (xk) ,

M k = {x : x~+~ E a (xk)},
m--t

k = O. . . . . m - - 1,

where x o is fixed.

It is clear that M =s
k=0

Mh and the optimal control problem posed reduces to the minimizatkj ,xj

tion 'of the function 7(x ~) =

%

f(x,~)on the set M.

It is clear that the derivative of f

%

with respect to x is the

vector with components {0 . . . . . 0, f~(x)}. We denote by K k the cone of possible directions at the point x~ towards the set M k. {P:p~EKo}, where Ko C X is the cone of vectors Pl such t h a t for s0me function o~(X) E X which satisfies the condition c~(~)~-'-+0 for X ~ 0.
K~ = {P: C~ Pk+,) E K~}, k = 1. . . . .

Clearly,

K0 =

x?+~.p,q-c~(~,)Ea(xo)for small X > 0 Analogously,

m - - 1, - x0 xo+i ) where K k is the cone o f possible directions toward graf a at the point z ~ - ( k,

THEOREM. conditions:

Suppose that the cones K k, k = 0, ..., m -

1, are convex and that we have the following

228

m--l.~

a) b) such that

the cone /~---(-I Ks is the cone o f possible directions for the set M at the point
m--l,~ 4=0 %

XO;

K*= ~Kk" "

k~0

Then for an optimal trajectory x ~ there exist linear functions x k * CX*,

k=

0, .... m,

(8)
z; Ea:k (x;+~), /e ----- m -- I ..... 1.

[4]

Proof. If x o is an optimal trajectory, then it is easy to see that a necessary condition for an e is the condition
f L (x~ E K',
x

(9)

if the condition a) of the theorem holds. detail the condition (9). It is not hard to see that

Therefore, for a p r o o f of the theorem it suffices to check in more

~ 2 = l~-* :(x;,x;+,)EK I, x ; = 0 ,
k= 1. . . . . m--l,

ivak, k + l},

Considering the expression for

f2 and the condition b) of the theorem, we get the relations ~;(x0) = x;~..... ~, 0
=

xL + xL~_,,

k = 1. . . . . m - - l ,

(10)

where
(x'k.~,x~+l.~)EK ~,

k = ! ..... m--l,

X~.oEK*o.

(1t)

We now use the notation

X ' ~ X ' k , k _ ~ , k = 1. . . . . m.

Then the relations (10) and (11) give the following:

xL = - x ; ,
(--x'k,x'k+:)EK'k, k = 1. . . . . m--

1,

r;(x0) = x~,

x~ c ~ .

From this, taking into account that K k is the cone o f possible directions to ~ a f a at the point z k - (x k, xk+~), we get fx (x0) = Xm,

x; C a;~ (x;+ O,
which concludes the p r o o f of the theorem.

k = 1. . . . .

m --

l,

Remark 1. If the mapping a(') is convex, i.e., graf a is a convex set, then in the case of a convex function f(x) the conditions are also sufficient. F o r this it is necessary to take as the cones o f admissible directions which define a~* the cones K~ = { p : H ~, > 0, such that (x ~Pl, Y + ~P2) E graf a}. In this case the condition a) o f the theorem is automatically fulfilled. Remark 2. There are various methods of verifying the condition b). ov In particular, if the cones K k intersect

at a point which is internal for all the cones, or if the space X if finite-dimensional, and K k are polyhedral, then the condition b) is fulfilled. Remark 3. The verification o f the condition a) is as a rule associated with the application of some implicit function theorems, but in a broad class o f cases this verification is relatively easily carried out. In particular, as an exercise it can easily be verified that the conditions a) and b) are fulfilled for the many-valued mapping defined in Examples I and 2 above. In both these examples (if the set U is polyhedral in Example 1) t~ the cones IZ~ are polyhedral.

229

Remark 4.

* E azk(Xk+ ) contains both an associated system for the maximum principle The condition x k

[3] and a condition for a local maximum. This is clearly obvious from the mapping defined in Example I. Indeed, the relations (3) and (4) in this case give x2 = (f; (~'
,rex"k~1' I ~" u'~-k' 1"~0 .0~/,~k -I 0~" ~

u~))'x~+l'

(12)

~Iulu "[1 (Xi'~-l' * t~ 9 (X~' "k0) " ) '

where u ~ x ~ are an optimal control and a trajectory, respectively, which coincides with the usual necessary condition for linear systems. We further observe in this example the role o f the last o f the conditions (8). this case we can take as the cone K 0 the cone K 0 = {p : p = [~ (x~, u 0 v, v E V,0}. Indeed, if p is an element of K 0, then from x ~ = f ( x o, % ) it follows that In

xO + ~,p + c~(~,) = f (Xo, Uo + ~.v),

where

i.e., for small X we have by definition o f implies that

V,,,u o -t-~,vEU and ~ + kp -4- a (~.) E a (x0). The fact that x* E K* now

i.e., fulfillment o f the local maximum principle also for the control at the zero step, while from the second relation of (8) the relation (12) followed only for k >/ 1. LITERATURE CITED 1. 2. 3. 4. 5. 6. V . G . Boltyanskii, Optimal Control with Discrete Processes [in Russian], Izd. Nauka, Moscow (1973). B . N . Pshenichnyi, "Convex multivalued mappings," Kibernetika, No. 3 (1972). B . N . Pshenichnyi, Necessary Conditions for an Extremum [in Russian], Izd. Nauka, Moscow (1969). A. Ya. Dubovitsldi and A. A. Milyutin, "Extremum problems with constraints," Zh. Vychisl. Mat. i Mat. Fiz., No. 3 (1965). V . G . Boltyanskli, "The support principle in problems of optimal control," Diff. Urav., 9, No. 8 (1973). R . T . Rockafellar, Convex Analysis, Princeton University Press (1970).

230