Chapter 7

Optimization Principles
7.1 Introduction
Optimization is the process of nding conditions which which give maximum or mini-
mum values of a particular function. It is important to understand that optimization
is done within the connes of a concept. We address trade os or give and take be-
haviour in a system to determine the best operating point. In some cases, even simple
objectives entail complicated analysis. We will focus on some simple methods which
can be applied to a host of engineering problems with limited use of mathematical
tools such as software.
7.1.1 The General Optimization Problem
In the mathematical statement of optimization, there is a single function:
y = f(x
1
, x
2
, x
3
, x
4
, x
4
...x
n
) (min/max) (7.1)
which is referred to as the objective function and the x
i
are the independent variables.
It is important that this function be stated as simply as possible. Often, it may require
several additional equations to dene variables which appear in the basic statement.
As an example, consider the optimization of a heat exchanger. The objective may
be to maximize the rate of heat transfer. Thus the objective function will involve
an expression which describes the heat transfer rate in a heat exchanger such as an
NTU relationship. This in turn requires the overall heat transfer coecient, which
is dened in terms of the uid stream heat transfer coecients, and so on.
The objective function may also be subject to constraints of the form:
g
i
(x
1
, x
2
, x
3
, x
4
, x
4
...x
n
) = 0 i = 1 . . . n
h
j
(x
1
, x
2
, x
3
, x
4
, x
4
...x
n
) 0 j = 1 . . . m
(7.2)
where g
i
represent equality constraints and h
j
represent inequality constraints. A
given problem may be unconstrained or constrained. We will examine both types of
143
144 Mechanical Equipment and Systems
problems and methods to solve them. It is also important to state the constraints in
a standard format as given above. This will be more evident when we examine the
Lagrange Multiplier method in a later section.
If a problem only contains linear equations, then it is often denoted as a linear
programming problem (LP), while if one or more equations in the problem statement
are non-linear, then it is referred to as a non-linear programming problem (NLP).
Unfortunately, many engineering optimization problems fall into the non-linear cate-
gory.
The eld of optimization is very broad. We will examine only select methods
which utilize calculus (gradient) methods using analytical, approximate analytical,
and numerical methods.
7.1.2 Basic Steps in an Optimization Problem
The following steps summarize the process of undertaking an optimization:
Analyze the process or system such that all of the specic characteristics and
variables are dened.
Determine the criterion for optimization and specify the objective function in
terms of the above variables.
Develop a mathematical model for the process or system that relates the in-
put/output variables using well known physical principles (mass, energy, mo-
mentum etc.) and empirical relationships.
If the problem is too large in scope, then break it up into smaller parts, or
simplify the objective and the model.
Apply a suitable optimization technique.
Check answers and examine sensitivity to changes in coecients and assump-
tions.
7.2 Unconstrained Optimization Problems and the
Gradient Method
Unconstrained optimization problems are the most simply stated. This is not to say
that they do not have constraints. It merely implies that the objective function and
any explicitly stated equality constraints have been back substituted into the original
objective function which leads to a new objective function which is now unconstrained.
This approach may not always possible, but attempts at it should always be made
rst. For these other types of problems we require additional methods to be discussed
in the next section.
Optimization Principles 145
In an unconstrained problem the principle result is that we must solve:
= f(x
1
, x
2
, x
3
, . . . x
n
) (min/max) (7.3)
The means to a solution is to consider the gradient of the function dened as:
= 0 (7.4)
The gradient of a scalar function is dened as the vector:
=
n

i=1

x
i
(7.5)
where
i
is a unit vector. If we consider a simple three dimensional function then the
gradient is simply:
=

i

x
+

j

y
+

k

z
(7.6)
A necessary condition for a minimum or maximum is that the gradient equal zero,
i.e. no slope! So a minimum would be a valley in three dimensional space, and a
maximum a hill. Therefore, we must nd a solution to the following set of equations:

x
1
= 0

x
2
= 0
.
.
.

x
n
= 0
_

_
(7.7)
We will now pursue some fundamental applications. These include but are not
limited to: optimization of pumping systems, the design of heat exchangers and
other devices, multi-stage compression, design of tanks and structures, etc. We shall
also consider other examples in the class notes.
7.3 Constrained Optimization Problems and
the Method of Lagrange Multipliers
The general theory for constrained multivariable optimization may be found in Reklaitis
et al. (1983), Edgar and Himmelblau (1988), Stoecker (1989) and Winston (1990).
The method of Lagrange multipliers may be easily applied to constrained multivari-
able applications. The general constrained Non-linear Programming (NLP) problem
takes the form:
= f(x
1
, x
2
, x
3
, . . . x
n
) (min/max) (7.8)
146 Mechanical Equipment and Systems
Subject to
g
j
(x
1
, x
2
, x
3
, . . . , x
n
) = 0 j = 1, . . . , m
h
j
(x
1
, x
2
, x
3
, . . . , x
n
) 0 j = m + 1, . . . , p
(7.9)
where g
j
and h
j
are imposed constraints. It is often more convenient to consider the
Langrangian form of the NLP in the following manner. A new objective function is
dened as follows:
L(x
1
. . . x
n
,
1
. . .
p
,
1
. . .
pm
) = (x
i
)+
m

j=1

j
g
j
(x
i
)+
p

k=m+1

k
(h
k
(x
i
)
2
k
) (7.10)
where
j
are Lagrange multipliers and
j
are slack variables. The use of slack variables
enables the Lagrange multiplier method to be applied to problems with inequality
constraints.
The problem is now reduced to solving the system of equations dened by
L
x
i
= 0 i = 1, . . . , n
L

j
= 0 j = 1, . . . , p
L

j
= 0 k = 1, . . . , p m
(7.11)
The above system may be solved using numerical methods such as a the mul-
tivariable Newton-Raphson method. The constrained formulation for NLPs with
inequality constraints can become quite complex. Given an NLP with n variables
and p constraints with p m inequality constraints, optimization of the Lagrangian
requires similtaneous solution of a system of n+2pm equations. In most problems,
the number of constraints prescribed should be judiciously chosen. For example, it
is not always necessary to prescribe that all x
i
> 0. In most problems, an optimal
solution with x
i
> 0 may be obtained if a reasonable initial guess is made while
leaving the particular x
i
unconstrained. While in other problems, constraints such
as x
i
< x
c
may not be necessary if the optimal solution returns x
i
< x
c
when x
i
are
unconstrained.
7.4 Newton-Raphson Method in Optimization
The Newton-Raphson method for a single variable was introduced in the last chapter.
The method is useful in optimization problems provided the objective function is twice
dierentiable.
Given an objective function f(x) and solution function:
g(x) =
df
dx
= 0 (7.12)
Optimization Principles 147
the solution for a variable x is found by making an initial guess x
i
and solving the
following equation:
x
i+1
= x
i

g(x
i
)
g

(x
i
)
= x
i

(x
i
)
f

(x
i
)
(7.13)
for the improved solution x
i+1
. The procedure is repeated until desired convergence
is achieved, which usually occurs in fewer than three iterations.
If our objective function is multivariable, the Newton-Raphson method can also
be applied. Given an objective function f(x
1
, x
2
, x
3
, x
4
, . . . , x
n
), we must solve the
system:
g
1
(x
1
, x
2
, x
3
, . . . x
n
) =
df
dx
1
= 0
g
2
(x
1
, x
2
, x
3
, . . . x
n
) =
df
dx
2
= 0
g
3
(x
1
, x
2
, x
3
, . . . x
n
) =
df
dx
3
= 0
.
.
.
g
n
(x
1
, x
2
, x
3
, . . . x
n
) =
df
dx
n
= 0
(7.14)
To solve the above system we may write the general non-linear system in a linearized
form using a truncated Taylor series expansion of each equation:
_

_
g
1
x
1
g
1
x
2

g
1
x
n
g
2
x
1
g
2
x
2

g
2
x
n
g
3
x
1
g
3
x
2

g
3
x
n
.
.
.
.
.
.
.
.
.
g
n
x
1
g
n
x
2

g
n
x
n
_

_
xo
_

_
x
1
x
2
x
3
.
.
.
x
n
_

_
=
_

_
g
1
g
2
g
3
.
.
.
g
n
_

_
xo
(7.15)
148 Mechanical Equipment and Systems
or in terms of the original objective function:
_

2
f
x
2
1

2
f
x
1
x
2


2
f
x
1
x
n

2
f
x
2
x
1

2
f
x
2
2


2
f
x
2
x
n

2
f
x
3
x
1

2
f
x
3
x
2


2
f
x
3
x
n
.
.
.
.
.
.
.
.
.

2
f
x
n
x
1

2
f
x
n
x
2


2
f
x
2
n
_

_
xo
_

_
x
1
x
2
x
3
.
.
.
x
n
_

_
=
_

_
f
x
1
f
x
2
f
x
3
.
.
.
f
x
n
_

_
xo
(7.16)
We may now solve the above above matrix as before, repeating iterations as required
until desired convergence of the solution is achieved. At each stage of iteration the
solution is rened according to:
x
i,new
= x
i,old
x
i
(7.17)
The matrix of second order partial derivatives is also referred to as the Hessian
matrix. It will be utilized in the next section to determine the optimality of a solution,
i.e. minimum, maximum, or saddle point.
7.5 Tests for Optimality
Up to this point we have not yet determined whether an obtained solution is indeed
maximum or minimum. In order to check whether or not a given solution is maximum
or minimum, we must return to the calculus of optimization.
7.5.1 Single Variable Functions
Given a single variable function f(x), whose optimal solution x = x
opt
satises:
df
dx

x=xopt
= 0 (7.18)
the test for optimality requires that we examine the value of the second derivative of
f(x), such that:
d
2
f
dx
2

x=xopt
0 min (7.19)
d
2
f
dx
2

x=xopt
0 max (7.20)
Optimization Principles 149
Further, if the second derivative is equal to zero:
d
2
f
dx
2

x=xopt
= 0 (7.21)
then we must consider further tests. Given a solution x
opt
which is referred to as
a stationary point, that is it satises the condition that f

(x
opt
) = 0, and the rst
non-zero higher order derivative denoted by n, i.e. f
n
(x) = d
n
f/dx
n
, then:
(i) If n is odd, then x
opt
is an inection point (neither min nor max)
(ii) If n is even, then x
opt
is a local optimum and if f
n
(x
opt
) > 0 then x
opt
is a local minimum or if f
n
(x
opt
) < 0 then x
opt
is a local maximum.
These are the necessary and sucient conditions for determining the state of the
stationary point.
7.5.2 Multi-variable Functions
In the case of a multi-variable function, the procedure is somewhat more complicated.
We must determine the eigenvalues of the Hessian matrix which is evaluated at the
stationary point (the potential optimal solution). Recall that the Hessian matrix is
dened as:
H(x) =
_

2
f
x
2
1

2
f
x
1
x
2


2
f
x
1
x
n

2
f
x
2
x
1

2
f
x
2
2


2
f
x
2
x
n

2
f
x
3
x
1

2
f
x
3
x
2


2
f
x
3
x
n
.
.
.
.
.
.
.
.
.

2
f
x
n
x
1

2
f
x
n
x
2


2
f
x
2
n
_

_
(7.22)
If we denote the stationary point of a multi-variable function:
x

= [x
1
, x
2
, , x
n
]
opt
(7.23)
then the eigenvalues of the Hessian matrix are determined by solving the following
equation:
det(H(x

) I) = 0 (7.24)
where are the eigenvalues and I is the identity matrix. The determinant of the
n n matrix yields an n
th
order polynomial equation in which has n real roots or
eigenvalues.
150 Mechanical Equipment and Systems
Once the eigenvalues have been determined, the following criteria determine whether
we have a minimim, maximum, or saddle point (multi-dimensional inection point).
If
i
are the eigenvalues of the Hessian matrix the the following apply:
i) H(x

) is positive denite if all

i
> 0 and the stationary point is a minimum
ii) H(x

) is positive semi-denite if all

i
0 with at least one
i
= 0, and
the stationary point is a minimum
iii) H(x

) is negative denite if all

i
< 0 and the stationary point is a maximum
iv) H(x

) is negative semi-denite if all

i
0 with at least one
i
= 0, and
the stationary point is a maximum
v) H(x

) is indenite if some
i
< 0 and
i
> 0, and
the stationary point is a saddle point
Optimization Principles 151
PROBLEMS
Problem 7.1
Consider the design of a large steel storage tank of diameter D and height H.
The goal is to maximize the volume V or capacity of the tank for a xed amount of
material (or surface area, A).
Solution
The mathematical statement of the objective function is:
V =
D
2
H
4
maximize (7.25)
subject to
A =
D
2
2
+ DH (7.26)
It should be stated that this is a relatively trivial example, since we have not really
said how the material of area A is being utilized. For example, if the material was a
large sheet, how are the ends and sides to be cut from this sheet?
We may combine the two equations to solve the problem as a simpler unconstrained
problem by writing the height H of the tank as:
H =
_
A
D
2
2
_
D
(7.27)
This gives the following objective function:
V =
AD
4

D
3
8
(7.28)
We can now nd the optimal diameter by taking the derivative of the objective
function, equating to zero, and solving:
dV
dD
=
A
4

3D
2
8
= 0 (7.29)
which gives:
D
opt
=
_
2A
3
(7.30)
and
H
opt
=
_
2A
3
(7.31)
The maximum volume of the tank can be found to be:
V
max
=

4
_
2A
3
_
3/2
(7.32)
152 Mechanical Equipment and Systems
The aspect ratio of the tank is H/D = 1. Additional constraints requiring the
aspect ratio to lie within a particular range could also have been specied.
We have just seen from Example 7.1, how to formulate an optimization problem.
In the optimization of engineering systems, many dierent objectives arise. These
include: minimize rst cost of a system, minimize life time costs, maximize prots,
minimize or maximize heat transfer, minimize pumping power, minimize weight, min-
imize or maximize work transfer, minimize surface area, maximize volume, minimize
pressure drop, minimize entropy generation, etc.
Problem 7.2
Re-examine the tank problem in Example 7.1. This time consider the volume to
be xed. We desire to obtain the system which has the least surface area A. This
could be for weight purposes or for heat transfer reduction. Formulate the objective
function for minimum surface area, minimum weight, and minimum cost.
Problem 7.3
Consider the function given by
f(x) = 5x
6
36x
5
+
165
2
x
4
60x
3
+ 36 (7.33)
Find all the stationary points and determine whether they are minimum, maximum,
or inection points.
Problem 7.4
Given the multi-variable function
f(x, y) =
2
xy
+ x
3
+ 4y
2
+ 2x
2
xy + 2x 4y (7.34)
determine the stationary point using Newton-Raphson iteration and determine whether
it is a minimum, maximum, or saddle point.
Problem 7.5 - Pumping System
Consider the classic problem of pumping uid between two points through a circular
pipe. We desire to nd the optimal pipe diameter which minimizes the total cost of
the constructing and operating the system, i.e.
C
T
= C
c
+ C
o
(7.35)
Assume that the cost of pump and piping is of the form:
C
c
= C
0
+ C
1
D
n
L (7.36)
and the operation cost of the system is of the form:
C
o
= C
2
mp

(7.37)
Optimization Principles 153
Further, assume that the pressure drop is calculated from:
p =
4fL
D
1
2
U
2
(7.38)
and that
f = 0.046Re
1/5
D
(7.39)
The constants C
0
, C
1
, and C
2
are cost indices for the pump, the piping, and the
electricity to run the pump. Modify the analysis to consider a pump cost which is
proportional to pipe diameter.
Problem 7.6 - Shell and Tube Heat Exchanger
A shell and tube heat exchanger requires a total length of tubing to be equal to
100 m based on heat transfer needs. The costs of the design of the heat exchanger
are as follows:
1) tube cost $900.00
2) shell cost $1100D
5/2
L
3) cost of space allocated to $320DL
Here D is the diameter of the shell and L is the length of the shell. The pre-selected
tube arrangement allows for up to 200 tubes/m
2
inside the shell cross-section. Find
the optimal D and L of the heat exchanger for minimum cost. Solve the problem as
an unconstrained problem. Use a conventional calculus approach and the intersection
of asymptotes method. Discuss the accuracy of the two solution approaches.
Problem 7.7 - Multi-stage Compression System
Consider a multi-stage compression system containing three compressors and two
intercoolers. The ideal work of each compression cycle (see Chapter 4), gives:
W
c
=
kRT
i
k 1
_
_
p
o
p
i
_
(k1)/k
1
_
(7.40)
If p
1
= 100 kPa and p
4
= 1000 kPa, what are the optimal values of p
2
and p
3
such
that the total work required is a minimum. Assume k = 1.4 for air and T
1
= 298 K.
Verify the solution is indeed a minimum.
Problem 7.8
Solve the following problem using Lagrange multipliers:
f = x
2
2y y
2
(7.41)
subject to:
g = x
2
+ y
2
1 0 (7.42)
154 Mechanical Equipment and Systems
Problem 7.9 - Shell and Tube Heat Exchanger
Re-solve Example 7.6 using the Lagrange multiplier method.
Optimization Principles 155
7.6 References
Arora, J.S., Introduction to Optimum Design,1989, McGraw-Hill, New York, NY.
Bejan, A., G. Tsatsaronis, and Moran, M., Thermal Design and Optimization,
1996, Wiley, New York, NY.
Beveridge, G.S.G. and Schechter, R.S., Optimization: Theory and Practice,
1970, McGraw-Hill, New York, NY.
Boehm, R., Design Analysis of Thermal Systems, Wiley, 1987.
Burmeister, L.C., Elements of Thermal Fluid System Design, 1997, Prentice-Hall,
Upper Saddle River, NJ.
Edgar, T.F. and Himmelblau, D.M., Optimization of Chemical Processes, 1988,
McGraw-Hill, New York, NY.
Jaluria, Y., Design and Optimization of Thermal Systems, 1998, McGraw-Hill, New
York, NY.
Rao,S.S., Engineering Optimization: Theory and Practice, 1996, Wiley, New York,
NY.
Reklaitis, G.V., Ravindran, A., and Ragsdell, K.M., Engineering Optimiza-
tion: Methods and Applications, 1983, Wiley, New York, NY.
Stoecker, W.F., Design of Thermal Systems, 1989, McGraw-Hill, New York, NY.
Winston, W.L., Operations Research: Applications and Algorithms, PWS-Kent,
1990, Boston.