Engineering design optimization

Lecture 10: Sensitivity analysis Multicriterion problems Response surface methods

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Lecture 10/10
P Calculation of gradients: sensitivity analysis <Analytical, numericaland semi-analytical methods <Stable perturbation intervals <Accuracyproblems P Multicriterion problems <Pareto-optimality <Bound formulation P Response surface methods < Numericalexperiments < The curse of dimensionality To Read: PArora:8.3 PErik Lund:Design sensitivity analysis of finite element discretized structures PJuhani Koski: Multicriterion structural optimization PRasmussen& Lund: An information management system for the formulation and solution ofmultidiscipinary optimization problems PMorten Johansen et.al.: Response surface approximations for shape optimization of a 3Dmanifold

Optimization methods
- almost always rely on gradients
Search methods (feasible directions, penalty, etc):

x( k + 1) = x(k ) + s( k ) s( k ) = f ( g i (x( k ) ))

Subproblem methods: gi (x) gi (x( k ) ) + g i (x( k ) ) (x x( k ) )+ ....

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Gradients
The gradient is a vector of partial derivatives

g g g g i = i , i , .. , i xn x1 x2
P If you know the analytical expressions for gi, then you can differentiate analytically. This is called the analytical method and is always the best and most efficient option, but unfortunately not possible in most cases. P If gi is purely numerically defined, then you can compute the gradient numerically. This takes time and introduces truncation and round-off errors, but it may be the only option. This is called overall finite difference . P If you know some things about the nature of gi but not the full expressions, then it may be possible to work analytically some of the way. This is called the semi-analytical method .

Overall Finite Difference, OFD

Forward finite difference Backward finite difference Central finite difference
g i g i (x + x j ) x j xj

g i g i (x x j ) x j xj

gi g i ( x + x j ) gi (x x j ) x j 2 x j

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Forward and backward FD

- are essentially the same
CentralFD We know that

g(x)

g(x+ )x)

g g x x for x 0
so we should use a small )x. The size, however is limited by the finite accuracy of the computer.

The cost of forward (or backward) FD is one additional analysis per design variable. The cost o f central FD is two additional analyses for each variable.

Accuracy considerations
Choosing the perturbation size
g x

Choosing the right value of )x is a compromise between round-off errors and truncation errors.
Stable interval

g x
Truncation errors

Round-off errors

)x Depending on the nature of the problem, the stable interval may be large, small or even non-existing.
J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Finite element analysis

- and all other types of numerical analysis based on solving systems of equations In finite element analysis, we roughly go through the following steps:
1. Form equations: K =

, f=

f +f
e

2. Factorize equations: K LDLT 3. Solve equations: LDLTu = f

If we denote the size of the problem in terms of nodes, ele ments or degrees of freedom by n, then step 1 is proportional to n, step 3 is propotional to n2, and step 2 is proportional to n3. So for some problem size, step 2 will always dominate.

Finite element Design Sensitivity Analysis

In FE analysis, we solve the problem:

Ku = f
Where K is a stiffness matrix, f is the load vector, and u is the vector of unknown displacements. We imagine that the problem depends on design variables x = { x1, x2, .. , xn}, and we want to find the sensitivity w.r.t. xj.

K u f u+ K = x j x j x j
We rearrange to find the displacement sensitivitites:

u f K = u x j x j x j

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Finite element DSA (contd)

The sensitivity equations have the same structure as the original equations:

Ku = f

Original problem Pseudo-load vector

u f K = u x j x j x j

We are solving a system with the same K as before, so we can re-use the same factorization. We do not have to perform the time-consuming operation K 6LDLT again!

Analytical and semi-analytical DSA

If we can compute

f K and xj x j
analytically, then we are performing analytical sensitivity analysis. If we compute

f f K K and x j x j x j x j
then we are using the so-called semi-analytical method. Notice that the FD computation of these terms is not very time consuming. It is only proportional to the problem size.

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Semi-analytical DSA in general

We found the sensitivities of the problem

Ku = f
Notice that this is not necessarily a finite element problem. This type of sensitivity analysis will apply to any numerical problem where the main operation is solution of a large system of equations. The method even applies if the system of equations is nonlinear, and the equations to find sensitivities still become linear. For large systems of equations, sensitivity analysis is much cheaper than ordinary analysis because we do not have to factorize again. This means that sensitivity analysis can be a very cheap way of getting a lot of information about the problem.

Accuracy in poorly conditioned problems

Lack of stable perturbation interval The condition number of matrix K is the difference between its largest and smallest eigenvalue. If the conditio n number is large, then the problem is ill-conditioned. This means physically that small changes of f can lead to large changes of the solution u in the problem Ku = f
So if f has small errors like in finite difference DSA, then the stable perturbation interval might not exist at all.

Exact result Round-off errors Truncationerrors )x

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Case Story
Semi-analytical DSA in auto design A young engineering intern at Ford in Michigan was given the assignment of DSA of a car body skeleton. He modeled the frame with beam elements and used the semianalytical method to find sensitivities. His results turned out to be completely wrong, but nobody could figure out why. In fact, the more accurately he modeled the frame, the more wrong were the results.

Case Story
Simple model example
The problem is easier to understand if we make a simpler example:

n beam elements M L

We wish to study the sensitivity of the end deflection w.r.t. the beam length, L. Notice that this problem is solved exactly within the Bernoulli-Euler theory for any number of elements, n. It is very simple to compute the analytical solution to this problem.
J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Case Story
Error dependency on number of elements
It turns out that the error grows with the square of the number of elements! Even a small number of elements makes the result completely wrong. The explanation is that beam problems are inherently ill-conditioned because they have both displacements and rotations in the u vector and both forces and moments in the f vector. But why does the problem get worse when we increase the number of elements?

Error

Case Story
Why does a large number of elements cause problems?
To understand why the problem is related to the number of elements, we must study the nature of the pseudo-load vector for the beam problem.
u f K K = u x j x j x j
Pseudo-load vector

The resulting tip deflection is the difference between the contributions from all the forces in different directions. The more elements, the more forces. If all the forces are a little bit wrong, then the resulting deflection is a small difference between two large erroneous numbers. J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Case Story
What are the possible solutions? There are basically two ways of eliminating this problem: 1. Eliminate errors in the pseudo-load vector. This means more analytical work. 2. Do not introduce FD errors until after the equatio n solution has been done -> overall finite difference.

Case Story
When to watch out for erroneous sensitivities
P When the numerical problem is ill-conditioned. This happens often if the physical problem has very different properties in different directio ns. A beam, for instance has a high axial stiffness and a lo w lateral stiffness. P When the problem involves properties with different dimensions, for instance
< Forces and moments < Displacements and rotations < Pressures and velocities

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Multicriterion Problems
- when you cannot decide
A well-posed design optimization problem has only one objective function. However, it can sometimes be difficult to decide which is the right one. In many cases, we have several properties that we would like to improve simultaneously. If we do so, we have not posed the problem correctly, and it will have many solutions. But it is a real issue in engineering design that you often cannot decide which is the best formulation before you see the final solutions.

What is a good bicycle?

- bicycle design must consider many different properties

P Stiffness P Lightness P Low cost P Durability

Flexibility

Any bicycle on the curve is optimal w.r.t. weight and flexibility. But which one is really the best?
J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Pareto optimality
A curve like this is called a Pareto curve. If we had three objectives instead of two, it would be a surface. For a multicriterion problem to be Pareto-optimal, is must fulfil the following requirement: It is not possible to decrease any objective function further without increasing at least one of the others.

Flexibility

Knowing the Pareto cur ve or sur face is extremely useful in practical design, but it can be difficult to obtain; each point on the curve corresponds to solving one optimization problem.

Multicriterion problem forulation

To solve a multicriterion problem we must form an objective functio n as a combination of all the multiple objectives of the original problem.

Minimize f(g 0 j (x )) Subject to g i (x ) Gi

Each different combination f() leads to a different point on the Pareto surface.

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Popular multicriterion combinations

1 n f (g 0 j ) = g 0 j n j =1
1 f (g 0 j ) = n

PMinimize the mean value: PMinimize the RMS: PMinimize the maximum value:

g
j=1

2 0j

f (g 0 j ) = max(g j 0 )

Min/max problems
Minimizing the maximum is an important applicatio n in many fields, for instance

174 MPa

111 MPa

PMinimizing the maximum stress in a structure to reduce the risk of material failure PMaximizing the minimum eigenfrequency to remove a vibration problem PMaximizing the minimum pressure in a flow to remove cavitation PMinimizing the maximum velocity in a flow to make it as uniform as possible PMinimizing the maximum actuation force in a machine to allow use of smaller and cheaper components

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Min/max problems
- bound formulatio n Min/max problems are very easy to solve because they can be bound formulated:
Minimize max( g 0 j ( x )) is converted to Minimize Subject to g 01( x) g 02 ( x) . . g 0n (x )

This usually creates a problem with many constraints, and it is best solved by a subproblem method, typically SLP. Convex programming based on dual methods will not work because there are too many constraints.

Response surface methods

- when sensitivity analysis is undesirable In some cases, sensitivity analysis can be very difficult or impossible:
P When there is no way to tamper with the analysis software P When there is no way to make sensitivity analysis efficient P When the problem has some degree of non-smoothness P When it is desired to include experimental data or experience into the formulation of the problem. In these cases we can use a response surface method. It works on 0'th order information only, so it just requires ordinary analysis.
J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Numerical Experiments
- in response surface methods. The basic idea of response surface methods is to conduct a number of (numerical) experiments and use the results to learn how the problem behaves. To do this efficiently requires planning. It is easier to understand the concept through the use of an example: Optimization of a diffuser (Jens I. Madsen et al.):

Optimize wall shape to minimize energy dissipation

Diffuser Optimization
- by response surface methods We parameterize the design boundary by a polynomial functio n:
y1 y2

y = a 4 x 4 + a3 x 3 + a2 x 2 where a4 = 1 1 1 y1 y2 + 2 4 18 5 1 a3 = y 1 + y 2 2 6 3 1 a2 = 3 y 1 y 2 + 4 9

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Diffuser Optimization
Feasible domain We want to constrain the problem to monotonic wall shapes. Having the polynomial function defined, it is possible to derive that the set of monotonic wall shapes form a near-elliptical regio n in the design space.
All the dots inside the feasible domain are places where we could potentially compute function values, i.e., loss of energy. But it is expensive. We assume that the objective function is a quadratic polynomial in y1 and y2, and we want to identify the coefficients. Statisticalmethods can be used to identify that outof the candidate points, the highlighted ones will determine the coefficients with best reliability. So we choose those 10 points and perform CFD analyses to find the energy loss.

Diffuser Optimization
Identified objective function

The identified objective function is analytical and the problem can be optimized by many methods, for instance in a spreadsheet. It turns out that the optimum is rather flat, so many shapes have almost the same energy dissipation.

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Response surface methods

The curse of dimensionality The biggest problem with response surface methods is that the number of necessary numerical experiments grows very quickly with the number of variables. We need at least quadratic functions to approximate the function(s) with. In 1D, a quadratic function requires at least 3 experiments In 2D, a quadratic function requires at least 6 experiments In 3D, a quadratic function requires at least 10 experiments In 10D, a quadratic functio n requires at least 66 experiments In nD, a quadratic function requires at least (n+1)(n+2)/2 experiments

Response Surface Methods

Qualities
P They can be used in cases where there are no sensitivities available. P They will smooth non-smooth problems and possibly allow optimization of them. P They provide analytical expressions for the behavior of the system and thereby build up a lot of useful information about the problem. P They allow information from other sources, for instance experiments or experience, to be included in the approximation.

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001

Response Surface Methods

Problems
P They can only be applied to problems with few design variables, perhaps up to 7 or 8, due to the curse of dimensionality. P Numerical experiments must be planned carefully. P The result depends on the ability to choose an approximation function that can emulate the behavior of the problem.

Assignments
Loose ends, examination, and a practical problem

Last chance to get help with previous assignments that you have not yet completed EMSD: Start defining your examination problem and let us discuss it. All others: Problem 8.8 (use a spreadsheet for the solution)

J ohn Rasmussen, I nstitute of Mechanical Engineering, Aalborg University, 2001