1

Optimization in Engineering Design

Mark Abramson, USAF Institute of Technology
Charles Audet, Gilles Couture,

Ecole Polytecnique. de Montreal

John Dennis, Rice University
Andrew Booker, Evin Cramer,
Paul Frank, Joerg Gablonski, Boeing Phantom Works
Thanks to: AFOSR, Boeing, LANL, SANDIA, ExxonMobil, NSF
IMA2003
2
Find the deepest point in a lake
Is it possible to promise the deepest point?
IMA2003
3
Not with a deterministic code
Global optimization is generally impossible
IMA2003
4
If we are lucky...
Let f(x) represent the depth at position x. Sometimes f
and at least its gradient are available so we can use local
models - eg assume that f(x) is a quadratic lake bottom
around the current best point
A quasi-Newton method:
Using information gathered so far, and assuming that the
lake bottom is quadratic, predict the deepest point
Find the depth and the gradient of the depth at the
predicted deepest point and update the depth model
IMA2003
5
Finite dierence derivatives?
If the gradient is not available, can we approximate each
partial NorthSouth and EastWest by nite dierences?
f(x)
x
NorthSouth

f(x + x
NorthSouth
) f(x)
x
NorthSouth
Maybe, but how to choose x is a serious problem
IMA2003
6
Most cases
What if the only way to measure the depth is to row out
there in 30 dimensions and drop a line?
Or, to wait days for a simulation of the depth to run
based on geophysics,topography, water temperature, etc
And for the resulting estimate to be pretty rough
And if you arent sure if depth is what you want to
measure
IMA2003
7
Conicting objectives in design
Performance Decisions
Payloads Decisions
Propulsion Decisions
Structures Decisions
Systems Decisions
Airplane Architecture
Airplane Level Architecture
IMA2003
8
Design Explorer Applications
Multidisciplinary wing
planform design
777 Engine Duct Seals
Helicopter Rotor Design
Machining,riveting,and drilling database
3-D Fighter Aerodynamics
Engine Nozzle
Performance
Shot peen forming of wing skins
Space Station Power System
Aerospike Nozzle
IMA2003
9
The target optimization problem
The design problem is:
minimize f
p
(x)
subject to x X {x
n
: C
p
(x) 0},
There are parameters and variables:
Contextual parameters p, supposedly xed, and
Optimization or design or control variables x
IMA2003
10
Properties of the target problem
p is xed in the simulation, but it is subject to
uncertainty because of incomplete knowledge (e.g.
material properties) or because the product is used
dierently (e.g., the altimeter is o by 5%)
x X must be satised when the underlying simulations
for f
p
, C
p
are called. X is simple, say a polygon
C
p
(x) 0 only has to hold at the solution
IMA2003
11
Properties of variables and constraints
Function evaluations typically involve numerically linking
legacy PDE solvers. Evaluations may fail unexpectedly
There are few correct digits. Accurate approximation of
derivatives is problematic
Some variables are categorical, i.e., the simulations
only run with certain discrete choices, e.g., choice and
sequencing of manufacturing processes - Lecture 3.
IMA2003
12
Uncertainty in design variables
Instead of specied design x

, the delivered product

might be the design specied by a nearby x
Very common in engineering design. Good data may be
tough to get
Robustify against such variability by giving up some
optimality at a given point in return for optimality over
nearby designs
IMA2003
13
Uncertainty in context variables
Products are designed for optimal performance under
specied operating conditions, loads, etc
We would be willing to give up some performance at the
exact conditions p in return for optimal performance over
all likely nearby conditions q - or we can constrain
probability of failure for the nominal p
Seems to be chance constrained formulation - nothing
new
IMA2003
14
Hubris or Sanguinity?
Both types of uncertainty can be modelled for
optimization by incorporation into f, C
minimize

f(x)
subject to x

X {x
n
:

C(x) 0},
where

f(x), c
j
(x) incorporate uncertainty modelled by
the users scheme of choice
Still belongs to target class, Bring em on for now, but
seek tighter coupling
IMA2003
15
A condence builder
Meckesheimer, Booker, and Torng report very good
performance of our approach on some structural
engineering problems in dealing with probability of failure
by constraints or as the objective
Reliability based design optimization using Design
Explorer available from Andrew.J.Booker@boeing.com
From this point on, assume formulation incorporates
uncertainty
IMA2003
16
More problem structure
Typically, f
p
(x) is actually

f
p
(x, u(x)), where u(x) are
ancillary variables gotten from some simulation with
input x
I.e., given x, solve F
p
(x, u) = 0 for u(x). Usually a
discretized PDE or several such legacy solvers. This is a
major reason why f
p
(x) fails to evaluate - the scheme to
couple the solvers fails to compute u(x)
These are called MDO problems, and they are hard...
IMA2003
17
Target to end 1st Lecture here
IMA2003
18
The target optimization problem
The design problem can be formulated:
minimize f(x, u(x))
subject to x X {x
n
: C(x) 0},
Optimization or design or control variables x
x X must be satised when the underlying simulations
for f, C are called. X is simple, say a polygon
C(x) 0 may include constraints on u(x)
IMA2003
19
Properties of variables and constraints
State or ancillary variables u(x) found by solving
F(x, u) = 0
Function evaluations typically involve numerically linking
legacy PDE solvers, so F(x, u(x)) = 0 can be
complicated
Solving F(x, u(x)) = 0 when there are multiple
disciplines is called a multidisciplinary analysis
IMA2003
20
An oversimple multidisciplinary analysis
Let
u
1
(x) = A
1
(x, u
2
(x)) or u
1
(x) A
1
(x, u
2
(x)) = 0
u
2
(x) = A
2
(x, u
1
(x)) or u
2
(x) A
2
(x, u
1
(x)) = 0
Newtons method would require dierentiating the PDE
solver codes A
i
w.r.t. their input - pray for automatic
dierentiation to succeed
IMA2003
21
MDA = Multidisciplinary analysis
January 3, 2003 6
D
1
Aerodynamics

?
E
12
?
M
1
M
12
A
1
?
U
1
F
21
?

21
-

D
2
Structures
-
?
E
21
?
M
2
M
21
A
2
?
U
2
F
12
?

12
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
Figure 1: Aeroelastic System
the so-called design point, like the viscosity of the air, angle of attack of the wing,
and speed. We have completed the MDA for a given decision vector x when we have
found u(x) such that the following hold for u = (M
1
, M
2
).
F
1
(x, M
1
, M
2
) = E
21
F
21
A
1
(M
1
) M
2
, (3)
F
2
(x, M
1
, M
2
) = E
12
F
12
A
2
(M
2
) M
1
. (4)
and A
1
is the the ow solver (maybe nite difference) and A
2
denotes the structures
code (maybe nite element). The E, F routines are interface routines that do such
bookkeeping as converting pressures to loads and deections to shape. Figure 2 pro-
vides a more detailed idea of what the E and F boxes look like in that there is probably
one required for each of the other disciplines connected to A
i
. These interface boxes
are the source of many of the difculties with MDO.
Why is successive replacements is used in MDA and not Newton? When faced with
a system of nonlinear equations to solve, we automatically think of Newtons method.
The problem is the availability of derivatives of F with respect to u. Each nite dif-
ference step would require a single discipline solve, and the number of correct digits
in the difference might be small anyway. Still, one suspects that sparsity is there to be
exploited, and the number of variables involved may be too large to use Broyden-type
updating methods. The most common approach is for the user to apply intuition to a
sequencing of the single discipline solvers and use successive replacements, i.e., x
u
2
, . . . u
n
s
, solve the rst equation block for a new u
1
. Now x all but u
2
with the
new u
1
, and use F
2
to get a new u
2
, etc, and iterate. If convergence is a problem, then
reorder the equation and unknowns and try again.
Formulations of MDO using aeroelastic design example
MDF or black box or closed equations or control theory method (different elds call it
different names. Different engineering disciplines are often different worlds.): This is
Disciplines D
1
, D
2
with analysis codes A
1
, A
2

ij
is compressed info from D
j
needed by D
i
IMA2003
22
Mathematical formulation of MDA
Write F(x, u) = 0, but solve F(x, M
1
, M
2
) = 0 for M,
where
F
1
(x, M
1
, M
2
) = E
21
F
21
A
1
(M
1
) M
2
,
F
2
(x, M
1
, M
2
) = E
12
F
12
A
2
(M
2
) M
1
A
1
nite dierence and A
2
nite element maybe
E, F are interface routines that do such bookkeeping as
converting pressures to loads and deections to shape
IMA2003
23
A generic discipline in an MDO problem
January 3, 2003 7
D
i
X
D
?

i1
?
E
i1
P
P
P
P
P
P
Pq
M
i1

i2
?
E
i2
Z
Z
Z~
M
i2

in
?
E
in

)
M
in

i1
?
E
i1

?
M
i
A
i
?
U
i

1i
?
F
1i

2i
?
F
2i

ni
?
F
ni
P
P
P
P
P
P
P q
Figure 2: Generic Discipline
the traditional approach in which a complete MDA for u(x) is done for each choice of
the design variables x. Thus, the optimization problem to be solved is a function only
of the design variables x, and it looks like (1). An important point is that the objective
function f(x) is almost surely of the form f(x) =

f(x, u(x)). The point being that the
state or system variables u are needed to evaluate the design objective function. Thus,
given x, we solve an MDA problem for u, and then we have the arguments to evaluate

f,

C. Notice that if there are bounds on u, then they become bounds on u(x), and hence
they are really nonlinear constraints. If bounds on u(x) must be closed, ( for example
bounds on u may represent safety considerations - like operating temperatures of some
chemical process), then we may have a nasty difculty for the optimization algorithm.
Advantage is optimization variables are the design variables (generally n
d
<< n
s
and every iterate can be made feasible. Thus, when we stop, we only need to
restore feasibility in C(x), often called the side constraints.
Disadvantage is computational effort to complete an MDA for every function
value. Even nite difference derivatives with respect to x require an MDA for
each forward difference quotient.
All-at-once (AAO) or Simultaneous Analysis and Design (SAND) or open equations
or nonlinear programming method: This is the ideal in many ways. It doesnt always
work. There is some analytic and practical evidence that nonsmooth behavior in the so-
lution u(x) gets smoothed as it passes through f(x, u(x)) and so the blackbox method
may work when the AAO method doesnt. However, when the AAO approach does
E, F routines are a major headache to write
IMA2003
24
The MDF or blackbox formulation
January 3, 2003 8
Optimizer (Controls calculation of f, C
D
)
?
X
D 6
U
1
, U
2
Aeroelastic Analysis Solver
D
1
Aerodynamics

?
E
12
?
M
1
M
12
A
1
?
U
1
F
21
?

21
-

D
2
Structures
-
?
E
21
?
M
2
M
21
A
2
?
U
2
F
12
?

12
D
D
D
D
D
D
D
D
D
D
D
D
D
Figure 3: Multidisciplinary Feasible (MDF) Method
Optimizer (Controls calculation of f, C
D
)
?
X
D
, X
U
1
, X
U
2
?
X
D
, X
U
1
, X
U
2
Aerodynamics
Residual
Computation
? -
W
1
6
Structures
Residual
Computation
?
W
2
6
Figure 4: All-At-Once or AAO Formulation
work, it seems most efcient. One solves the AAO problem as:
min
x,u
f(x, u) s.t. x X, F(x, u) = 0, and C(x) 0 . (5)
Notice that this requires that we open the single discipline solvers in order to be
able to evaluate their internal residuals ( remember, U is produced by solving an inter-
nal system of equations). This is expensive, and generally it is resisted by the single
discipline specialists.
Advantage is less expensive function values and derivatives as well as a generally
less nonlinear optimization problem
Disadvantage is many more variables (the design variables + the ancillary vari-
ables u, as well as the inability to use legacy disciplinary solvers. Feasible point
algorithms are not compatible
Requires a full MDA for every evaluation
Derivatives must be through solvers - requires
du
dx
IMA2003
25
The All-At-Once or SAND formulation
January 4, 2003 8
Optimizer (Controls calculation of f, C
D
)
?
X
D 6
U
1
, U
2
Aeroelastic Analysis Solver
D
1
Aerodynamics

?
E
12
?
M
1
M
12
A
1
?
U
1
F
21
?

21
-

D
2
Structures
-
?
E
21
?
M
2
M
21
A
2
?
U
2
F
12
?

12
D
D
D
D
D
D
D
D
D
D
D
D
D
Figure 3: Multidisciplinary Feasible (MDF) Method
Optimizer (Controls calculation of f, C
D
)
?
X
D
, X
U
1
, X
U
2
?
X
D
, X
U
1
, X
U
2
Aerodynamics
Residual
Computation
? -
U
1
6
Structures
Residual
Computation
?
U
2
6
Figure 4: All-At-Once or AAO Formulation
work, it seems most efcient. One solves the AAO problem as:
min
x,u
f(x, u) s.t. x X, F(x, u) = 0, and C(x) 0 . (5)
Notice that this requires that we open the single discipline solvers in order to be
able to evaluate their internal residuals ( remember, U is produced by solving an inter-
nal system of equations). This is expensive, and generally it is resisted by the single
discipline specialists.
Advantage is less expensive function values and derivatives as well as a generally
less nonlinear optimization problem
Disadvantage is many more variables (the design variables + the ancillary vari-
ables u, as well as the inability to use legacy disciplinary solvers. Feasible point
algorithms are not compatible
(x, u) = (x
d
, x
u
) all are optimization variables, and
F
p
(x
d
, x
u
) = 0 is enforced only in the limit
Cant use legacy solvers, but derivatives are simpler -
du
dx
avoided
IMA2003
26
The IDF or DAO formulation
January 3, 2003 9
Optimizer (Controls calculation of f, C
D
, C
12
, C
21
)
?
X
D
, X

12
?
X
D
, X

21
Aerodynamics
Analysis
? -
F
21
(X
D
, U
1
(X))
(including, e.g., U
1
)
6
Structures
Analysis
?
F
12
(X
D
, U
2
(X))
(including, e.g., U
2
)
6
Figure 5: Individual Discipline Feasible or IDF Formulation
Individual Discipline Feasible (IDF) or In-Between methods: Generally, a small per-
centage of the state variables from one disciplines solver are needed as input to other
solvers. Suppose we reect this by writing u = (z, u), and z is the smaller subvector
of the solver variables u actually needed as input to various solvers. We can now pose
the MDO problem as:
min
x,z

f(x, z, u(z, x)) s.t. x X, C(x) 0, (z, u(z, x)) u(x) = 0. (6)
Advantage is ability to use legacy disciplinary solvers as they are and the opti-
mization problem is nearer the size of the MDF than the AAO problem. Seems to
be more efcient than the blackbox.
Disadvantage is that the derivatives needed are like those needed for MDA, and
feasible point algorithms are not compatible
Alexandrov and Lewis seem to have rediscovered this formulation, although there may
be some difference we missed, and they do not compare it to IDF. They call it DAO
or distributed analysis optimization, a nicer name than IDF [3]. Alexandrov and
Kodiyalam give computational evidence that this is an effective formulation [2].
MDA only in the limit - many fewer variables than AAO
Uses legacy solvers, but requires
du
dx
IMA2003
27
Summary
Blackbox/MDF is least ecient, but most stable, and
most often used in practice. Derivatives can sometimes
be gotten by adjoint approaches, but not if there are
many constraints
AAO is most ecient when it works, but incompatibility
with legacy codes is a huge barrier to practical use in
most elds Very common in chemical process control
IDF/DAO seems a good compromise - generalizes
domain decomposition
IMA2003
28
Summary
Many designers mistrust a method like AAO or IDF that
doesnt have all feasible iterates
I got the idea for IDF from Georg Bocks direct multiple
shooting
Blackbox or MDF will be used in my Math Coll.
numerical results for the SMF, which is derivative-free
A heuristic called collaborative optimization mimics
the by hand traditional approach
IMA2003
29
DAKOTA project at SANDIA attacks same problem
class by more heuristic approaches
Automatic dierentiation is the critical technology for
MDO
IMA2003