Professional Documents
Culture Documents
(2.1)
2.1
Lagrangian Sufficiency
The following result provides a condition under which minimizing the Lagrangian,
subject only to the regional constraints, yields a solution to the original constrained
problem. The result is easy to prove, yet extremely useful in practice.
Theorem 2.1 (Lagrangian Sufficiency Theorem). Let x X and Rm such that
L(x, ) = inf x X L(x , ) and h(x) = b. Then x is an optimal solution of (1.1).
Proof. We have that
min f(x ) = min[f(x ) T (h(x ) b)]
x X(b)
x X(b)
> min
[f(x ) T (h(x ) b)]
x X
2.2
x1 x2 2x3
subject to x1 + x2 + x3 = 5
x21 + x22 = 4.
The Lagrangian of this problem is
L(x, ) = x1 x2 2x3 1 (x1 + x2 + x3 5) 2 (x21 + x22 4)
= (1 1 )x1 2 x21 + (1 1 )x2 2 x22 + (2 1 )x3 + 51 + 42 .
For a given value of , we can minimize L(x, ) by independently minimizing the
terms in x1 , x2 , and x3 , and we will only be interested in values of for which the
minimum is finite.
The term (2 1 )x3 does not have a finite minimum unless 1 = 2. The terms
in x1 and x2 then have a finite minimum only if 2 < 0, in which case an optimum
occurs when
L
= 1 1 22 x1 = 3 22 x1 = 0 and
x1
L
= 1 1 22 x2 = 1 22 x2 = 0,
x2
i.e., when x1 = 3/(22 ) and x2 = 1/(22 ). The optimum is indeed a minimum, because
2 L
2 L
2
0
2
x x
x1 x2
,
=
HL = 12 1
2
L
L
0
22
x2 x1
x2 x2
is positive semidefinite when 2 < 0.
Let Y be the set of values of such that L(x, ) has a finite minimum, i.e.,
Y = { R2 : 1 = 2, 2 < 0}.
For every Y, the unique optimum of L(x, ) occurs at x () = (3/(22 ), 1/(22 ), x3 )T ,
and we need to find Y such that x () is feasible to be able to apply Theorem 2.1.
Therefore,
9
1
x21 + x22 = 2 + 2 = 4
42 42
p
2.1 to conclude
that the minimization
and thus 2 = 5/8. We can now use Theorem
p
p
problem
p has an optimal solution at x1 = 3 2/5, x2 = 2/5, and x3 = 5 x1 x2 =
5 + 4 2/5.
Let us formalize the strategy we have used to find x and satisfying the conditions
of Theorem 2.1 for a more general problem. To
minimize f(x) subject to h(x) 6 b, x X
(2.2)
we proceed as follows:
1. Introduce a vector z of slack variables to obtain the equivalent problem
minimize f(x) subject to h(x) + z = b, x X, z > 0.
2. Compute the Lagrangian L(x, z, ) = f(x) T (h(x) + z b).
3. Define the set
Y = { Rm : inf xX,z>0 L(x, z, ) > }.
4. For each Y, minimize L(x, z, ) subject only to the regional constraints, i.e.,
find x (), z () satisfying
L(x (), z (), ) = inf xX,z>0 L(x, z, ).
(2.3)
2.3
Complementary Slackness
implies
i = 0
i 6= 0
implies
(z ())i = 0.
and
Indeed, if the conditions were violated for some i, then the value of the Lagrangian
could be reduced by reducing (z ())i , while maintaining that (z ())i > 0. This
would contradict (2.3). Further note that Y requires for each i = 1, . . . , m either
that i 6 0 or that i > 0, depending on the sign of bi . In the case where where i 6 0,
we for example get that
(h(x ( )))i < bi
i < 0
implies
i = 0 and
implies
(h(x ( )))i = bi .
Slack in the corresponding inequalities (h(x ( )))i 6 bi and i 6 0 has to be complementary, in the sense that it cannot occur simultaneously in both of them.
x1 3x2
x1 3x2
HL =
2 L
x1 x1
2 L
x1 x2
2 L
x2 x1
2 L
x2 x2
21
0
0
21