Lecture 15 :

Matrix
Completion ,
Correlated
Random Variables ,
and

Deconvolution
A Matrix
.

Completion
"
P
X ER ( movies customers )
eg . n ,
p

assume X is low rank

subset of entries of X
only observe

want to fill in remainder

s.tt#MF*=X
r={
*
.
*
Cij )
*

Xij is observed }
* * *
* -
*
 I

ref
t
st
Mij=Xij Cider
=

argnmnin
.

an

mail.tjaetaw.TL#Dg*oFo03
^X=
argmmin
HMH *
= trace norm

= nuclear norm

V. (
sit .

Mij =Xj per

=
-

Computationally tractable
Iterative Singular Value Threshold
ing
initialize : ^X =
Zeros ( n , p )

In Xr fill obs entries

=
in .

set threshold or r
= =

for 1<=1 ,
2
,
...

[u ,
s ,v]= svd ( I ) ;

= S .
* ( S > threshold )
oe

l=l ^5 ( ,r)=S( )
(n ) ;
for
r r
=
zeros
,p ,
...
,r , ,
r
;

I =
V. 5. VT ;
K had
He 's
Xr
-

if
converged =Xr= , stop
end
Deconvolution
have signal of interest WERP
W

I
m -
i
p

we obs blurred version :

Yi =mtoWi+j +
Ei
A
y
-

-mMtwmn
-
noise

YERP

p
 define XERMP s .t XW + E
y
=
.

=3
e.g. m

X
1 10 o " "
is
1

Cireulant
X zt
=

o 1 l i 0 0 0

00
each row is
001 1 1

shifted Version of
0 00 1
1
10

0 0 0 0 1 1 1 previous row

( ]
1 0 0 0 0 1 I

l I 0 0 0 0

To est w
,
iv. ( XTXYX 'y
More formulation
general :

Yi=hg
define XERPM
.

|
y=Xw+e
st .

-
Convolution -7

Wisnjnesptq
h he h hp
X
- -
-

, }
,

hp h , hz - - -

hp .

, Circular

hp .

, hp h ,
. -
'

hp .
2

: h }
ha .
.
-
. h ,
 "
X =

VAV ( VH means
"
Hermitian
"

transpose ( )*
't
V =
V
when X circnlanti
VHV =] )
then

diagonal etts A Fourier transform

of coeffs of h
.
=

V is Fourier transform matrix

Fourier transform represent
:
signals as

Weighted sums of sinusoids


nnw
-
~
+ mmm
+
Muunnmrvm
 Fourier trends form of h

i*"ci%__ E.
a hi we
fonnsgoansitne
=

,
wave

A twin Hp
y h e-
=

i.

Is
'
ii. e
'

fp . h freq
=

;
,

k"
=pgA it
"

h ; ,
frequency
sample

( write h as weighted sum of sinusoids )

 These sinusoids correspond to the columns of V !

(k " ) %
kth of 's "p
v. < col
,fo[et2" eitnck
=

eiza ftp.D.gg
=

. . .

Xp ]
h =
V A D= 41 dz
, , ,
...

,
,

"

Hutt "
kth
s[jtaa' %ae.jzack.in/p...jjzaa#5f
=

row of

-
.

a.
=
vi. h

"

D= V h
y=w= .
VI.
vans

of
't

ligen decomposition X VAV

=

.
diagetts of A =
Xk 's .

XHX Signal processing

Consider LS est of W :

"Hy perspective

.lv/wayY=w
iv. ( )
:

|"
* h tax
=( VAVHVAVD

EE
?,
'

?
"
""
is
,
inter
VHY DFT Y= dft ( y )
1 1

y
=
. .

2. scale DFT caeffs 2 .

H= dft ( h) ,
=
Y ./H
, by dividing by DFT
coeffof h

3 . inverse DFT 3 .
in =
idft ( W )
 (
Consider Ridge Regression
I .

dftcy )

iv. ( vNv*tHI'v^v*y
=( ( ii. MIVHYVAVHY
defeats
(
)
avoid
dividing
bywos
3. inverse dft

Vy
aha
=
-

~ Some dk 's Close to

I
-

zero

xitx Xz

* .

'

Xp
'

- if .
Multivariate normal 1 Gaussian distribution
p ( )

# 'Ez
'

mail.q.es
>
*

# :*
.
x
Xz X

.ie?ieIIIIII
,
 %)
=[0j
# 2 ,
E

#ia .

,
'

{ x ER s .t .

1111=8 }
 g ;]
=[0j
p -2 .
E

# /
E
F

{ xeR2 }
"
:
xt [ =
y

x-p
F.
+
1 =
g
 [
10=2 ,

arbitrary positive definite matrix .

xeR2 }
"

{ XTE x=8
:

factorsFx
=

'll "x '

=y

z.tv#xMxxtV/ilVtx
let - Xz

xi rotation defined

'
,
,

T spread
defined by
eigenvalues .
Mike
x
>

,