Professional Documents
Culture Documents
Multidimensional Unconstrained
Optimization
Part 8
Jia Pan
Office: AC-1 Y6720
Email: jiapan@cityu.edu.hk
Analytical Method
For one dimensional unconstraint optimization
problem, the optimal point is the point where
the curve is flat
In mathematical terms, the first derivative f(x)=0
If second derivative f(x)<0, the point is a maximum
If second derivative f(x)>0, the point is a minimum
MBE2036 Engineering Computing, part 8, version 2
H= Eq 1
Example 1
Find the optimal point of f(x,y)=x2 y2
0 as x and y are
independent variables
Similarly,
MBE2036 Engineering Computing, part 8, version 2
f(x,y)=x2 y2
0.3
0.2
0.1
0
z
-0.1
-0.2
-0.3
-0.4
0.5
0
0.5
0
y
-0.5
-0.5
x
MBE2036 Engineering Computing, part 8, version 2
Example 1(cont)
=2
From Eq 2:
=
Example 2
Find the optimal point of f(x,y)=2xy +2x x2 2y2
2xy x2 2y2
=
2xy x2 2y2
= y Eq 4
MBE2036 Engineering Computing, part 8, version 2
10
-10
-20
Z
-30
-40
-50
-60
-2
-1
0
-2
1 -1
0
2
1
3 2
3
Y 4 4 X
MBE2036 Engineering Computing, part 8, version 2
-10
100 -20
0
Z
-100
-2
-30
-1
-2
0 -1
-40
1 0
1
2
2 -50
3
3
Y X
4 4
MBE2036 Engineering Computing, part 8, version 2
Example 2 (cont)
To find the optimal point:
4y =0 Eq 6
Example 2 (cont)
4y =0
=-2 4y
4y
2
MBE2036 Engineering Computing, part 8, version 2
Example 2 (cont)
=
Gradient Methods
In any multi-dimensional optimization searching
algorithm, there are two key questions to be
asked when searching for an optimal solution.
1. The first one is about the direction of travel.
2. The second question is how long one should pursue
a solution in this direction before re-evaluating an
alternative direction.
MBE2036 Engineering Computing, part 8, version 2
Directional derivative
of f(x,y) at point x=a
and y=b
MBE2036 Engineering Computing, part 8, version 2
For n dimensions
f
x ( x )
1
f ( x )
f ( x) x 2 Eq8
f
( x )
x n
MBE2036 Engineering Computing, part 8, version 2
f
x xo h Eq 9
x
f
y yo h Eq 10
y h is distance
along the h axis
MBE2036 Engineering Computing, part 8, version 2
Example 3
Using Steepest Ascent method for finding the maximum
point of f(x,y)=2xy +2x x2 2y2
2xy x2 2y2
2xy x2 2y2
Recall:
is the
= y Eq 4 max point
MBE2036 Engineering Computing, part 8, version 2
Example 3 (cont)
Using initial guesses, x=-1 and y=1
y= 2(1) 4(1)=6
Therefore the gradient vector is:
MBE2036 Engineering Computing, part 8, version 2
Example 3(cont)
To find the maximum, we could search along the
gradient direction, that is, along an h axis running
along the direction of the above gradient vector.
Example 3(cont)
Multiplying the terms, we develop a one-dimensional
function along the gradient as:
Therefore
This means that If we travel along the h0-axis, g(h)
reaches maximum value when h=0.2
MBE2036 Engineering Computing, part 8, version 2
Example 3(cont)
Substitute h=0.2 into Eq 9 and Eq 10
Example 3(cont)
MBE2036 Engineering Computing, part 8, version 2
Example 3(cont)
y= 2(0.2) 4(0.2)=1.2
MBE2036 Engineering Computing, part 8, version 2
Example 3(cont)
f(x,y)=2xy +2x x2 2y2
Substituting x2, y2 values into the functions yields:
Therefore
This means that If we travel along the h1-axis, g(h)
reaches maximum value when h=1
MBE2036 Engineering Computing, part 8, version 2
Example 3(cont)
= 0.2+1.2(1)=1.4
Example 3(cont)
MBE2036 Engineering Computing, part 8, version 2
Example 3(cont)
y= 2(1.4) 4(1)=-1.2
MBE2036 Engineering Computing, part 8, version 2
Example 3(cont)
What can you observe?
MBE2036 Engineering Computing, part 8, version 2
Seems
Is this a coincidence?
MBE2036 Engineering Computing, part 8, version 2
Other Methods
Newtons method
Taylors series
Q&A
Thank you