Lagrange

Lagrange Multipliers
Multipliers

Optimization in Engineering Design

Georgia Institute of Technology

Systems Realization Laboratory

Lagrange
Lagrange Multipliers
Multipliers
The method of Lagrange multipliers gives a set of necessary conditions to identify
optimal points of equality constrained optimization problems.
This is done by converting a constrained problem to an equivalent unconstrained
problem with the help of certain unspecified parameters known as Lagrange
multipliers.
The classical problem formulation
minimize
f(x1, x2, ..., xn)
Subject to

h1(x1, x2, ..., xn) = 0

can be converted to
minimize
L(x, ) = f(x) - h1(x)
where
L(x, v) is the Lagrangian function
is an unspecified positive or negative constant called the Lagrangian Multiplier
Optimization in Engineering Design

Georgia Institute of Technology

Systems Realization Laboratory

Finding
Findingan
an Optimum
Optimum using
usingLagrange
Lagrange
Multipliers
Multipliers
New problem is:

minimize L(x, ) = f(x) - h1(x)

Suppose that we fix = * and the unconstrained minimum of L(x; )

occurs at x = x* and x* satisfies h1(x*) = 0, then x* minimizes f(x)
subject to h1(x) = 0.
Trick is to find appropriate value for Lagrangian multiplier .
This can be done by treating as a variable, finding the unconstrained
minimum of L(x, ) and adjusting so that h1(x) = 0 is satisfied.

Optimization in Engineering Design

Georgia Institute of Technology

Systems Realization Laboratory

Method
Method
minimize L(x, ) = f(x) - h1(x)

1.

Original problem is rewritten as:

2.

Take derivatives of L(x, ) with respect to xi and set them equal to

zero.

If there are n variables (i.e., x1, ..., xn) then you will get n equations with n + 1
unknowns (i.e., n variables xi and one Lagrangian multiplier )

3.

Express all xi in terms of Langrangian multiplier

4.

Plug x in terms of in constraint h1(x) = 0 and solve .

5.

Calculate x by using the just found value for .

Note that the n derivatives and one constraint equation result in n+1
equations for n+1 variables!

(See example 5.3)

Optimization in Engineering Design

Georgia Institute of Technology

Systems Realization Laboratory

Multiple
Multipleconstraints
constraints
The Lagrangian multiplier method can be used for any
number of equality constraints.
Suppose we have a classical problem formulation with k equality constraints
minimize
f(x1, x2, ..., xn)
Subject to

h1(x1, x2, ..., xn) = 0

......
hk(x1, x2, ..., xn) = 0

This can be converted in

minimize
where

L(x, ) = f(x) - T h(x)

T is the transpose vector of Lagrangian multpliers and has length k

Optimization in Engineering Design

Georgia Institute of Technology

Systems Realization Laboratory

In
Inclosing
closing
Lagrangian multipliers are very useful in sensitivity
analyses (see Section 5.3)
Setting the derivatives of L to zero may result in finding
a saddle point. Additional checks are always useful.
Lagrangian multipliers require equalities. So a
conversion of inequalities is necessary.
Kuhn and Tucker extended the Lagrangian theory to
include the general classical single-objective nonlinear
programming problem:
minimize
Subject to

f(x)
gj(x) 0 for j = 1, 2, ..., J
hk(x) = 0 for k = 1, 2, ..., K
x = (x1, x2, ..., xN)

Optimization in Engineering Design

Georgia Institute of Technology

Systems Realization Laboratory