Professional Documents
Culture Documents
Lecture 6 Duality
61
Duality theorem
notation p is the primal optimal value; d is the dual optimal value p = + if primal problem is infeasible; d = if dual is infeasible p = if primal problem is unbounded; d = if dual is unbounded
duality theorem: if primal or dual problem is feasible, then p = d moreover, if p = d is nite, then primal and dual optima are attained note: only exception to p = d occurs when primal and dual are infeasible
Duality
63
Weak duality
lower bound property: if x is primal feasible and z is dual feasible, then c T x bT z proof: if Ax b, AT z + c = 0, and z 0, then 0 z T (b Ax) = bT z + cT x cT x + bT z is the duality gap associated with primal and dual feasible x, z weak duality: the lower bound property immediately implies that p d (without exception)
Duality 64
Strong duality
if primal and dual problems are feasible, then there exist x, z that satisfy c T x = bT z , Ax b, A T z + c = 0, z 0
combined with the lower bound property, this implies that x is primal optimal and z is dual optimal the primal and dual optimal values are nite and equal: p = c T x = bT z = d (proof on next page)
Duality
65
0 A
=c
the lower-bound property implies that any solution necessarily satises c T x + bT z = 0 to prove a solution exists we show that the alternative system (p. 55) u 0, t 0, AT u + tc = 0, Aw tb, bT u + c T w < 0
has no solution
Duality
66
the alternative system has no solution because: if t > 0, dening x = w/t, z = u/t gives z 0, AT z + c = 0, Ax b, cT x < bT z
this contradicts the lower bound property if t = 0 and bT u < 0, u satises u 0, A T u = 0, bT u < 0
if the dual problem is feasible and z is any dual feasible point, then z + tw 0, AT (z + tw) + c = 0 for all t 0
therefore z + tw is dual feasible for all t 0; moreover, as t , bT (z + tw) = bT z tbT w + so the dual problem is unbounded above
Duality 68
if the primal problem is feasible and x is any primal feasible point, then AT (x + ty ) b for all t 0
therefore x + ty is primal feasible for all t 0; moreover, as t , cT (x + ty ) = cT x + tcT y so the primal problem is unbounded below
Duality 69
Duality
610
PSfrag
Summary
p = + d = +
primal inf. dual unb. optimal values equal and attained exception primal unb. dual inf.
p nite
p =
d nite
d =
upper-right part of the table is excluded by weak duality rst column: proved on page 68 bottom row: proved on page 69 center: proved on page 65
Duality 611
Outline
Variants
LP with inequality and equality constraints minimize cT x subject to Ax b Cx = d standard form LP minimize cT x subject to Ax = b x0 maximize bT y subject to AT y c maximize bT z dT y subject to AT z + C T y + c = 0 z0
dual problems can be derived by converting primal to inequality form same duality results apply
Duality 612
Piecewise-linear minimization
minimize f (x) = max (aT i x + bi )
i=1,...,m
Interpretation
for any z 0 with
i zi
= 1, for all x
this provides a lower bound on the optimal value of the PWL problem min f (x) min z T (Ax + b)
x x
bT z if AT z = 0 otherwise
the dual problem is to nd the best lower bound of this type strong duality tells us that the best lower bound is tight
Duality 614
-Norm approximation
minimize LP formulation minimize subject to dual problem maximize bT u + bT v subject to AT u AT v = 0 1T u + 1T v = 1 u 0, v 0 simpler equivalent dual maximize bT z subject to AT z = 0,
Duality
Ax b
t A 1 A 1 x t b b
(1)
(2)
615
proof of equivalence of the dual problems (assume A is m n) if u, v are feasible in (1), then z = v u is feasible in (2):
m
=
i=1
|vi ui| 1T v + 1T u = 1
if z is feasible in (2), dene vectors u, v by ui = max{zi, 0} + , with = (1 z 1)/(2m) these vectors are feasible in (1) with objective value bT (v u) = bT z vi = max{zi, 0} + , i = 1, . . . , m
Duality
616
Interpretation
lemma: uT v u
1
1,
Ax b
z T (Ax b)
this provides a bound on the optimal value of the -norm problem min Ax b
x
min z T (Ax b)
x
bT z
if AT z = 0 otherwise
the dual problem is to nd the best lower bound of this type strong duality tells us the best lower bound is tight
Duality 617
Outline
Optimality conditions
primal and dual LP minimize cT x subject to Ax b Cx = d maximize bT z dT y subject to AT z + C T y + c = 0 z0
optimality conditions: x and (y, z ) are primal, dual optimal if and only if x is primal feasible: Ax b and Cx = d y , z are dual feasible: AT z + C T y + c = 0 and z 0 the duality gap is zero: cT x = bT z dT y
Duality
618
Complementary slackness
assume A is m n with rows aT i the duality gap at primal feasible x, dual feasible y , z can be written as c T x + bT z + d T y = (b Ax)T z + (d Cx)T y = (b Ax)T z
m
=
i=1
z i ( bi a T i x)
aT zi = 0 i x < bi =
619
Geometric interpretation
example in R2
a1 x c a2
in other words, c = a1z1 + a2z2 with z1 0, z2 0 geometric interpretation: c lies in the cone generated by a1 and a2
Duality 620
Example
4x 1 5x 2 1 0 2 1 subject to 0 1 1 2 show that x = (1, 1) is optimal second and fourth constraints are active at (1, 1) therefore any dual optimal z must be of the form z = (0, z2, 0, z4) with 2 1 1 2 z2 z4 = 4 5 , z 2 0, z4 0 minimize
x1 x2
0 3 0 3
z = (0, 1, 0, 2) satises these conditions dual feasible z with correct sparsity pattern proves that x is optimal
Duality 621
Optimal set
primal and dual LP (A is m n with rows aT i ) minimize cT x subject to Ax b Cx = d assume the optimal value is nite
> 0} let (y , z ) be any dual optimal solution and dene J = {i | zi
maximize bT z dT y subject to AT z + C T y + c = 0 z0
Strict complementarity
primal and dual optimal solutions are not necessarily unique any combination of primal and dual optimal points must satisfy z i ( bi a T i x ) = 0, in other words, for all i, aT i x < bi , z i = 0 or aT i x = bi , z i > 0 or aT i x = bi , z i = 0 i = 1, . . . , m
primal and dual optimal points are strictly complementary if for all i aT i x < bi , z i = 0 or aT i x = bi , z i > 0
it can be shown that strictly complementary solutions exist for any LP with a nite optimal value (exercise 72)
Duality 623