EE263 Autumn 2015 S. Boyd and S.

Lall

Extremal-trace problems

1
Maximizing and minimizing quadratic forms
given n × n symmetric matrix A
maximize : xT Ax
subject to : kxk = 1
eigenvalue decomposition of A:
n
X
λi qi qiT
i=1

I λ1 ≥ · · · ≥ λn eigenvalues of A

I q1 , . . . , qn ∈ Rn orthonormal eigenvectors

I solution: x = q1

I optimal value: λ1

to minimize:

I solution: x = qn

I optimal value: λn
2
Maximizing and minimizing sums of quadratic forms

Pk
maximize : i=1xTi Axi
subject to : kxi k = 1
xTi xj = 0 i 6= j
compact representation:

maximize : Tr(X T AX)

subject to : X TX = I

I solution: x1 = q1 , . . . , xk = qk

I optimal value: λ1 + · · · + λk

to minimize:

I solution: x1 = qn , . . . , xk = qn−k+1

I optimal value: λn + · · · + λn−k+1

3
Maximizing the input gain

given A ∈ Rm×n Pk 2
maximize : i=1 kAxi k
subject to : kxi k = 1,
xTi xj = 0 i 6= j
extremal-trace formulation:
maximize : Tr(X T AT AX)
subject to : X TX = I

with variable X ∈ Rn×k

equivalent ways of expressing the solution:

qk , where q1 , . . . , qk are eigenvectors of AT A

 
I X = q1 ···
 
I X = v1 ··· vk , where v1 , . . . , vk are right singular vectors of A

4
Application: worst-case earthquake

simple model of building during earthquake:

x(t + 1) = Ax(t) + Bu(t), x(0) = 0,

y(t) = Cx(t)

I u is ground displacement

I y is displacement of top of building

I problem data:
  

0.9 0.5 1  
A= , B= , and C= 1 2
−0.5 0.7 −1

I assumptions about input:

Tu
X
ku(t)k2 ≤ 1 and u(t) = 0, t > Tu
t=0

where Tu = 50
5
Application: worst-case earthquake

find worst-case earthquake according to criterion

T
X
ky(t)k2 ,
t=1

where T = 100:
maximize : kM uk2
subject to : kuk2 ≤ 1,
where
···
 
CB 0 0 0
 CAB CB 0 ··· 0 
 2 
M =  CA
 CAB 0 ··· 0 

 .. .. .. .. .. 
 . . . . . 
CAT −1 B CAT −2 B CAT −3 B ··· CAT −Tu −1 B

6
Application: worst-case earthquake
solution: first right singular vector of M

0.5
u(t)

−0.5
0 20 40 60 80 100

5
y(t)

−5
0 20 40 60 80 100
t
7
Application: worst-case earthquake

find worst-case earthquake according to criterion ky(T )k2 :

maximize : kM̃ uk2

subject to : kuk2 ≤ 1,

where

M̃ = CAT −1 B CAT −2 B CAT −3 B CAT −Tu −1 B

 
···

8
Application: worst-case earthquake
solution: first right singular vector of M̃

0.5
u(t)

−0.5
0 20 40 60 80 100

5
y(t)

−5
0 20 40 60 80 100
t
9
Minimizing the projection error

given A ∈ Rm×n
Pn Pk
minimize : j=1 k(I − i=1 xi xTi )aj k2
subject to : kxi k = 1,
xTi xj = 0 i 6= j

where aj is jth column of A

equivalently, maximize the projection power
Pk Pn T 2
maximize : i=1 j=1 (xi aj )
subject to : kxi k = 1,
xTi xj = 0 i 6= j

10
Minimizing the projection error

extremal-trace formulation:
maximize : Tr(X T AAT X)
subject to : X TX = I

with variable X ∈ Rn×k

equivalent ways of expressing the solution:

qk , where q1 , . . . , qk are eigenvectors of AAT

 
I X = q1 ···
 
I X = u1 ··· uk , where u1 , . . . , uk are left singular vectors of A

11
Application: total least squares
data points (xi , yi ) ∈ R2 for i = 1, . . . , n: yi = a(xi + δi ) + b + i , a = b = 1

12

10

6
y

0
0 2 4 6 8 10 12
x
12
Application: least squares

least squares chooses â and b̂ to minimize

n
X
(âxi + b̂ − yi )2
i=1

solution:  †  
  x1 1 y1
âls  .. ..   .. 
= . .  . 
b̂ls
xn 1 yn

13
Application: total least squares
least squares: âls = 0.70, b̂ls = 2.97; relative error in (âls , b̂ls ) = 1.99

12

10

6
y

0
0 2 4 6 8 10 12
x
14
Application: total least squares

total least squares chooses x0 , y0 , and d to minimize

n     2
X x0 xi
min + t i d −
ti ∈R y0 yi
i=1

subject to kdk = 1
solution:
n n
1X 1X
x0 = x̄ = xi , y0 = ȳ = yi
n i=1 n i=1
 
x1 − x̄ ··· xn − x̄
d = first left singular vector of
y1 − ȳ ··· yn − ȳ

recover slope and intercept:

d2
âtls = and b̂tls = ȳ − âtls x̄
d1

15
Application: total least squares
total least squares: âtls = 1.07, b̂tls = 0.98; relative error in (âtls , b̂tls ) = 0.07

12

10

6
y

0
0 2 4 6 8 10 12
x
16