Professional Documents
Culture Documents
Optimization
Lagrange Multiplier
M S Prasad
This lecture note is based on Text book and open literature suitable course Engineering system design
Optimization .It should be read in Conjunction with class room discussions.
=0
and
=0
h1(x) d1 ... hp(x) dp. We have both equality and Inequality constraints .
we know that we can always transform an Inequality constraints as an Equality constraints by
introducing a slack variable (a non negative parameter ) i.e gi ( x) 0 is equivalent to
gi (X) + si (0) where S 0. By this process we have introduced one more variable and one
more constraint S 0. To solve this problem we use S2i i.e gi + S2i = 0
The S are calculated by making sure that j* 0 for J = 1 .m. also partial derivative of S is
equated to zero.
We form a Lagrangian as under
( , , ) = () + () + ( () + 2 )
1
We calculate
= 2 ..
1
=0
=0
it means { g ( x*) + S 2} = 0
And
= 0 => 2 i* Sj = 0 known as Complimentary condition .
and
check i* 0 that is non negativity test.
These condition are known as KKT i.e. Karush Kuhn Tucker( KKT) necessary and sufficiency
condition .
In general, the Lagrangian is the sum of the original objective function and a term that involves
the functional constraint and a Lagrange multiplier . Geometric meaning of Lagrange
Function is that at minimum point ,gradient of cost function and constraint functions are
along the same line and they are proportional to each other by .
Since L(x* , *) = 0.
Hence f(x* ) + h(x* = 0.
Think of as knob that we can turn to adjust the value of x.
Imagine turning this knob until we find a value of , say = , such that the functional
constraint is satisfied, i.e., g(x()) = b. Let x = x(). Our claim is that x solves P. This is the socalled Lagrangian Sufficiency Theorem.
Lagrange multiplier for Quadratic constraints
The most general form of constraint optimization problem is formulated as under :-
where x has dimensions , nx1 , f(x) is the objective function to be minimized, g( x) are a set of
inequality constraints, and h(x) are a set of equality constraints. Inequality constraints of the form
can be rewritten as g(x) 0 To solve numerically the following standard forms are chosen
f(x) = xT A X + bTX +c
g(x) = Dx e
h(x) = Cx d
= + + = 0
= = 0
} { } = [ ]
minimize P (x)
Now the problem is formulated as unconstrained optimization. However, we cannot solve directly
because large values of can cause instability and inefficiency when deriving a solution with high
accuracy. We can use the sequential unconstrained minimization technique (SUMT) to incrementally
increase the penalty parameter as we derive the solution incrementally.
1. Choose tolerances 1 , 2 very small such as 10 -5, starting point x0 =0, initial penalty
parameter 0=1
2.
1 and
2 then
---------------------------------------------------------------------------------------------------------