4 views

Uploaded by Danny Chamorro

numerico

- Intro 1.0
- Global Image Denoising
- F1DB5d01
- Engineering Mathematics Cheat Sheet
- 0601048
- Pac
- ERP Multicriteria Decision Making
- Final Exam
- An even order symmetric B tensor is positive definite
- hs5
- Lecture 3
- untitled.pdf
- FREQUENCY RESPONSE ANALYSIS OF 3-DOF HUMAN LOWER LIMBS
- Maxima Book Chapter 7
- Coeficientes de Influencia
- 363 _3_
- Linear Maps
- Autonomous Vision Based Facial and voice Recognition on the Unmanned Aerial Vehicle
- Gij07EV
- pcaLectureShort_6pp

You are on page 1of 11

Guangri Xue

Problem 1 (4.1). Let f (x) = 10(x2 x21 )2 + (1 x1 )2 . At x = (0, 1) draw the contour

lines of the quadratic model

1

mk (p) = fk + fkT p + pT Bk p (1)

2

assuming that B is the Hessian of f . Draw the family of solutions of

1

minn mk (p) = fk + fkT p + pT Bk p s.t. kpk k (2)

pR 2

Solutions: Compute

f = , 2 f = .

20(x2 x21 ) 40x1 20

0, and B + I 0.

At x = (0, 1),

T 42 0

fk = 11, fk = [2, 20] , Bk = .

0 20

T

2 20

(Bk + I)p = fk , thus p = , .

42 + 20 +

If = 0, kp k = 442/21. If > 0, kp k < 442/21.

[1/21, 1]T , 442/21 2.

p = h iT

2

42+ 20

, 20+ , with kp k = , 0 < < 442/21.

When 0 < < 442/21, we use Newton method to find . See the Figures 1 and 2 for

the contours and the solutions.

1

Figure 1: Contour of mk at x = (0, 1) with Figure 2: Zooming in the left Figure 1

solutions

Figure 3: Contour of mk at x = (0, 0.5) with Figure 4: Zooming in the left Figure 3

solutions

At x = (0, 0.5),

7 18 0

fk = , fk = [2, 10]T , Bk = .

2 0 20

T

2 10

p = , , with kp k = , 0 2.

18 + 20 +

See the Figures 3 and 4 for the contours and the solutions.

2

Problem 2 (4.2). Write a program that implements the dogleg method. Choose Bk to

be the exact Hessian. Apply it to solve Rosenbrocks function

Experiment with the update rule for the trust region by changing the constants in Algo-

rithm 4.1, or by designing your own rules.

Solutions:

400x1 (x2 x21 ) 2(1 x1 ) 400x2 + 1200x21 + 2 400x1

2

f (x) = , f (x) = .

200(x2 x21 ) 400x1 200

= 0.1 and = 103 .

When x0 (1.2, 1.2)T . The following is the convergence history.

1.94e+01 2.29e+02 1.00e+00 0

1.41e+01 1.92e+02 5.27e+00 1

6.26e+00 1.24e+02 7.88e+00 2

4.06e-02 9.15e+00 6.22e+00 3

1.96e-03 1.45e-01 3.87e-02 4

3.46e-04 8.33e-01 1.61e-03 5

9.23e-08 8.86e-04 3.46e-04 6

8.53e-13 4.13e-05 9.23e-08 7

5.64e-25 2.24e-12 8.53e-13 8

For this initial guess, the hessian matrix is always positive definite (checked through the

code) during this iterative process. So dogleg method should work well. In fact x0 is close

to the exact solution. As a result the method behaves like Newtons method. In addition,

kf (x)k decreases monotonically.

With = 0 and = 0.1,

5.80e+00 1.25e+02 1.00e+00 0

3.11e+00 9.01e+01 2.69e+00 1

2.65e-01 2.49e+01 2.85e+00 2

1.41e-02 3.03e-01 2.51e-01 3

6.26e-03 7.86e-01 7.89e-03 4

1.54e-03 8.56e-01 4.71e-03 5

1.29e-04 3.27e-01 1.41e-03 6

3

1.60e-06 3.03e-02 1.28e-04 7

3.26e-10 5.50e-04 1.60e-06 8

1.39e-17 8.98e-08 3.26e-10 9

One can see that kf (x)k does not decrease monotonically, which verifies the convergence

theory numerically when = 0.

We test the method with a more conservative data: = 0.01 and = 105 . It took

31 iterations instead of 8 before.

Now we take x0 = (1.2, 1)T . In this case, the hessian is not always positive definite.

However, the iteration converges in 190 iterations with = 0.1 and = 103 . Of course,

we can not guarantee that kf (x)k decreases monotonically.

4

Problem 3 (4.6). The Cauchy-Schwartz inequality states that for any vectors u and v,

we have

|(u, v)|2 (u, u)(v, v)

with equality only when u and v are parallel. When B is positive definite, use this inequality

to show that

kgk4

:= 1

(g, Bg)(g, B 1 g)

with equality only if g and Bg (and B 1 g) are parallel.

Proof. First of all, B must be symmetric. If not, the following is a counter example. Take

1 1

1 1

B= . Then B 1 = 2 2 .

1 1 21 12

Let g = [g1 , g2 ]T . Then, (g, Bg) = g12 + g22 and (g, B 1 g) = 12 (g12 + g22 ). As a result

= 2 > 1.

So B is symmetric positive definite. Let 0 < 1 2 n be the eigenvalues.

Pn And

xi , (i = 1, 2, ..., n), are the corresponding orthonormal eigenvectors. Write g = i=1 ci xi .

We have

Xn Xn X n

2 1 1 2

(g, Bg) = i ci , (g, B g) = i ci , (g, g) = c2i .

i=1 i=1 i=1

By Cauchy-Schwartz inequality,

n

!2 n

!2 n n

1

X X 1 X X

c2i = (i ci )(i 2 ci )

2

i c2i 1 2

i ci

i=1 i=1 i=1 i=1

5

Problem 4 (4.7). When B is positive definite, the double-dogleg method constructs a

path with three line segments from the origin to the full step. The four points that define

the path are

the origin;

(g,g)

the unconstrained Cauchy step pC = (g,Bg) g;

a fraction of the full step pB = B 1 g, for some (, 1], where is defined in

previous question; and

the full step pB = B 1 g.

Proof.

pC ,

0 1

p( ) = C B C

p + ( 1)(p p ), 1 2

( + ( 2)(1 )) pB , 2 3

When 1 2, let h() = 21 kp(1 + )k2 .

1 2

h() = kpC k2 + (pC , pB pC ) + kpB pC k2 .

2 2

It is enough to show h0 () > 0 for (0, 1).

(g, g) 1 (g, g)

= g, B g + g

(g, Bg) (g, Bg)

(g, g) 1 (g, g)

= (g, B g) (g, g)

(g, Bg) (g, Bg)

(g,g)

Since > , we have (g, B 1 g) (g,Bg) (g, g) > 0. This implies h0 () > 0 for (0, 1).

When 2 3, it is obvious kp( )k is monotonically increasing function.

6

Problem 5 (4.9). Derive the solution of the two-dimensional subspace minimization

problem in the case where B is positive definite.

Solutions:

1

min m(p) = f + g T p + pT Bp kpk , p span[g, B 1 g]

p 2

Let p = g + B 1 g and u = (, )T .

1

J(u) := kpk2 = 2 kgk2 + 2 kB 1 gk2 + 2(g, B 1 g) = uT Bu,

2

here

kgk2 (B 1 g, g)

B = 2 .

(B 1 g, g) kB 1 gk2

Easy to see B is symmetric positive definite.

We derive the constraint for u from kpk .

J(u) 2 . (3)

1

h(u) = f + (g + B 1 g, g) + (g + B 1 g, Bg + g)

2

2 2

= f + kgk2 + (B 1 g, g) + (g, Bg) + (B 1 g, g) + kgk2

2 2

T 1 T

= f + g u + u Bu,

2

here

kgk2 kgk2

(Bg, g)

g = , B =

(B 1 g, g) kgk 2 (B 1 g, g)

Therefore the problem reduces to solve the following two dimensional problem: under

the constraint (3),

min h(u). (4)

uR2

Easy to check B is symmetric positive and definite. Let u be the minimizer of (4) with

no constraint.

u = B 1 g.

If J(u ) 2 , then u is the solution.

If J(u ) > 2 , solve the following problem: for 0,

h(u) + J(u) = 0

7

This is equivalent to

(B + B)u = g

(B + B) is symmetric positive definite. Then the solution is

(B + B)1 g.

u , J(u ) 2

argmin h(u) =

(B + B) g, J(u ) > 2

1

J (B + B)1 g = 2 .

8

Problem 6 (4.8). Show that

2

2 ((l) )

(l+1) (l) (l+1) (l) kpl k kpl k

= 0 (l) , and = +

2 ( ) kql k

are equivalent.

Proof: From the hint, we have

d 1 1 3 d

02 () = ( ) = (kp()k2 ) 2 kp()k2 , (5)

d kp()k 2 d

and

n

X (g T g)2

d

kp()k2 = 2 . (6)

d (j + )

j=1

n

X (g T g)2

kqk = . (7)

(j + )

j=1

Note that

1 1

2 () ( kp()k )

0 = 3 ,

2 () 12 (kp()k2 ) 2 d

d

kp()k2

by (7), we have

2

kp()k3 (kp()k )

2 () kp()k kpk

0 = = .

2 () kqk2 (kp()k) kqk

9

Problem 7. Consider the problem the problem (4.5) and its approximate solutions:

pD : approximation obtained by the dogleg method;

pT : solution of the two dimensional problem (4.17); and

p : exact solution of (4.5)

Prove that

m(p ) m(pT ) m(pD ) m(pC ).

Proof:

pT , pD , pC S . Since the p is the minimizer of (4.5) over S , then we have

m(p ) m(pT ), m(p ) m(pD ) and m(p ) m(pC ).

, p span{g, B 1 g}}. Since pC is the minimizer of (4.5) along the path p =

kgk g, 0 < 1 and pD is the minimizer of (4.5) along the path

T

D

ggT Bg

g

g 0< 1

p ={ Tg gT g

ggT Bg g + ( 1)(B 1 g + g T Bg

g) 1 < 2.

gT g

Let pU = g T Bg

, which is the minimizer of (4.5) without any constrains. (i) If kpU k >

gT g kgk3

, then we have g T Bg

kgk . Hence g T Bg

1. This means pC = kpU k

pU and

pC is on the path of dogleg. Since pD

is the minimizer along the dogleg path, we

have m(p ) m(p ). (ii) If kp k , it is easy to see that pD = pC = pU . Hence

D C U

m(pD ) = m(pC ).

10

Problem 8. The symmetric matrix B has eigenvalues 1 2 n with corre-

sponding orthonormal eigenvectors qi , i = 1, 2, , n. When q1T g 6= 0 (the hard case in the

textbook), consider

X qT g

i

p( ) = qi + q 1 .

i 1

i6=1

P qiT g

Proof: According to the definition p = i6=1 i 1 qi + q1 . Then we have

X qiT g X (q T g)2

kp( )k2 = k qi + q1 k2 = i

+ 2.

i 1 (i 1 )2

i6=1 i6=1

Since

X (qiT g)2

2 ,

(i 1 )2

i6=1

v

(qiT g)2

u X

= t2

u

(i 1 )2

i6=1

11

- Intro 1.0Uploaded byhakes
- Global Image DenoisingUploaded byZhiqian Lu
- F1DB5d01Uploaded byRauf Khan Rind
- Engineering Mathematics Cheat SheetUploaded bytevin sessa
- 0601048Uploaded byKrishnaveni Subramani S
- PacUploaded byfull entertainment
- ERP Multicriteria Decision MakingUploaded byPurdianta Yo
- Final ExamUploaded bySara Gallego
- An even order symmetric B tensor is positive definiteUploaded bySumit Kushwaha
- hs5Uploaded byAbdel Daa
- Lecture 3Uploaded bynarendra
- untitled.pdfUploaded bySean Pierre
- FREQUENCY RESPONSE ANALYSIS OF 3-DOF HUMAN LOWER LIMBSUploaded byJames Moreno
- Maxima Book Chapter 7Uploaded byprakush_prakush
- Coeficientes de InfluenciaUploaded byRicardo Colosimo
- 363 _3_Uploaded bysummer
- Linear MapsUploaded byTom Davis
- Autonomous Vision Based Facial and voice Recognition on the Unmanned Aerial VehicleUploaded byEditor IJRITCC
- Gij07EVUploaded byRio Ardian
- pcaLectureShort_6ppUploaded byCarl Wells
- 5.2.1.nullingantenna.pdfUploaded bySridhar Sai Reddy
- Todd Kapitula, P.G. Kevrekidis and R. Carretero-Gonzalez- Rotating matter waves in Bose–Einstein condensatesUploaded byLomewcx
- 9231_s02_erUploaded byAlif Azizi
- Training AI to defendUploaded byDavide Gallitelli
- 0712 3060v1Uploaded byJay Liew
- Spin 1Uploaded bymierko
- Ch2Laplace.pdfUploaded byJhill-Jhill Jimenez Dela Peña
- Numerical Solution of Singular Eigenvalue Problems for ODEsUploaded byGuilherme Morais
- Discret IzationUploaded byem01803257
- Handbook of Thin Plate Buckling and PostbucklingUploaded byReivax50

- ejercicios_listasUploaded byDanny Chamorro
- AnalisisNumerico1Uploaded byDanny Chamorro
- Proyecto de OptimizacionUploaded byDanny Chamorro
- Antologia MusicalUploaded byMauro Coronel
- 2. Energia Sistemas Cerrados (1)Uploaded byDanny Chamorro
- Deber OptimizacionUploaded byDanny Chamorro
- Esque LasUploaded byDanny Chamorro
- Clase_derivacion_integracion_funciones_2014.pdfUploaded byPablo Antonio Zapata
- TIPOS_DE_MATRICES_ESPECIALES.pdfUploaded byDanny Chamorro
- TIPOS_DE_MATRICES_ESPECIALES.pdfUploaded byDanny Chamorro
- Lista Cosas Por ComprarUploaded byDanny Chamorro
- Cultura General 8vo. de EGB N° 6Uploaded byDanny Chamorro
- DocumentoUploaded byDanny Chamorro
- ejempoUploaded byDanny Chamorro
- Fundamentos de MatematicaUploaded byDanny Chamorro
- TEORIA_DE_CAMPOS.pdfUploaded byChristian Chavez Pereyra
- proyectoUploaded byDanny Chamorro
- Sistema de ReferenciaUploaded byAmIn20122
- Manual - HP49GUploaded bysal sal
- Tensor Analysis SokolnikoffUploaded byIndhujah Somasundaram
- deber de quimicaUploaded byDanny Chamorro
- Chapter1_7thUploaded byPrarit Lamba
- Complement Os Mate Mati CosUploaded bymikallita88
- Paganini, Nicolo - Caprices, Violin (1-24)Uploaded byDanny Chamorro
- MAN DR Ejemplos LagrangianaUploaded bycarlosnovoan
- Tema7_Polarizacion-1x2Uploaded byDanny Chamorro
- Estructuras y Base de DatosUploaded byDanny Chamorro
- c++ (Peter Class)Uploaded byzinpaiz

- Act 438 Free Zones Act 1990Uploaded byAdam Haida & Co
- CRS China's Currency---A Summary of the Economic IssuesUploaded bychina economy
- Prevailing Wage Request Form for IowaUploaded byRolando Fuentes
- Artist Avenue 2018 BrochureUploaded bymj
- Right of First RefusalUploaded byTracie Sangalang
- SNL0MB0010709Uploaded byJune C Straight
- EBA Guidelines Version 3 2015Uploaded byziehonk
- V40232 Clark C270 Torque ConverterUploaded byRemHenry
- Carbonate Sedimentology and Sequence StratigraphyUploaded byMirza Ali
- (Music Reference Collection) Larry Sitsky-Australian Piano Music of the Twentieth Century -Praeger (2005).pdfUploaded byAhmad
- AP Physics B Free Response IndexUploaded byandrew4mail94
- Descriptive StoryUploaded bykathrynlynne6
- Acor Hdpe GuideUploaded byKautik Shah
- CS_SHRMUploaded byzskebapci
- APC ST1131 Sample FinalUploaded byGary Ong
- spawning networkUploaded byashwinikapse134891
- Web Ready Proxy_Costco from Donnelly 2015 - Final.pdfUploaded byHarpreet Matharoo
- Final Cut Pro X HelpUploaded byDon Stalter
- Holiday Guide for the Region of HildesheimUploaded bymeinNiedersachsen
- H28-2AUploaded byjwsigler
- CELTA TP2 LANGUAGE ANALYSISUploaded bymsslittlesunshine
- Cotton TRAAUploaded byram
- 7-Eleven User Profile Migration Project2Uploaded byJoseph Nichols
- ARIBA GuideUploaded bySatyabrata Nath
- ACTG101_-_BSA_syllabus_2nd_sem_15-16.docUploaded byJelmond Duffergrey Ragudo
- [Braindump2go] 2015 Latest 74-409 Dumps Free Download (41-50)Uploaded byDemi Davison
- Rashays Menu SEP2016 WebUploaded byAmna Pervaiz
- Work Energy PowerUploaded byHemant Chaudhari
- science has been working in order to find answersUploaded byapi-219303558
- Oracle Financials Cloud 2018 PreSales Specialist 15QsUploaded byabebe