Professional Documents
Culture Documents
The Basics
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Every column
of D is a
prototype
Introduction
The Basics
Conclusions
Every column
of D is a
prototype
Similar to, but
more general
than, PCA
Introduction
The Basics
Conclusions
Introduction
The Basics
Sparse Coding in V1
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Outline
Conclusions
Introduction
The Basics
Outline
Conclusions
Introduction
The Basics
Outline
Conclusions
Conclusions
Introduction
The Basics
Outline
Conclusions
Conclusions
Introduction
The Basics
Outline
Conclusions
Conclusions
Introduction
The Basics
n
X
i=1
i di = D
Conclusions
Introduction
The Basics
n
X
i di = D
i=1
Learning:
Given training data xj , j {1, , m}
Learn dictionary D and sparse code
Conclusions
Introduction
The Basics
n
X
i di = D
i=1
Learning:
Given training data xj , j {1, , m}
Learn dictionary D and sparse code
Encoding:
Given test data x, dictionary D
Learn sparse code
Conclusions
Introduction
The Basics
j=1
i=1
Conclusions
Introduction
The Basics
j=1
i=1
j=1 i=1
Conclusions
Introduction
The Basics
j=1
i=1
2
subject to kdi k c,
j=1 i=1
i = 1, , n.
Conclusions
Introduction
The Basics
j=1
j=1 i=1
i=1
2
subject to kdi k c,
i = 1, , n.
In matrix notation:
arg minkX ADk2F +
D,A
subject to
X
|i,j |
i,j
X
i
D2i,j c,
i = 1, , n.
Conclusions
Introduction
The Basics
j=1
j=1 i=1
i=1
2
subject to kdi k c,
i = 1, , n.
In matrix notation:
arg minkX ADk2F +
D,A
subject to
X
|i,j |
i,j
D2i,j c,
i = 1, , n.
Conclusions
Introduction
The Basics
i = 1, , n.
Conclusions
Introduction
The Basics
Conclusions
i = 1, , n.
j=1
!
X
i
Di,j c
Introduction
The Basics
Conclusions
i = 1, , n.
j=1
!
X
i
Di,j c
Introduction
The Basics
Conclusions
j=1
!
X
i
D2i,j c
Introduction
The Basics
Conclusions
j=1
!
X
i
D2i,j c
Introduction
The Basics
Conclusions
j=1
!
X
D2i,j c
Introduction
The Basics
Conclusions
j=1
!
X
D2i,j c
Introduction
The Basics
Conclusions
j=1
!
X
D2i,j c
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
X
i,j
|i,j |
Conclusions
Introduction
The Basics
|i,j |
i,j
Conclusions
Introduction
The Basics
Conclusions
|i,j |
i,j
Introduction
The Basics
Conclusions
|i,j |
i,j
Introduction
The Basics
Conclusions
Introduction
The Basics
Outline
Conclusions
Conclusions
Introduction
The Basics
Outline
Conclusions
Conclusions
Introduction
The Basics
Dictionaries are
over-complete bases
Conclusions
Introduction
The Basics
Dictionaries are
over-complete bases
Dictate relationships
between atoms
Conclusions
Introduction
The Basics
Dictionaries are
over-complete bases
Dictate relationships
between atoms
Example: Hierarchical
dictionaries
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Problem Statement
Goal:
Have sub-groups of sparse code
all be non-zero (or zero).
Introduction
The Basics
Conclusions
Problem Statement
Goal:
Have sub-groups of sparse code
all be non-zero (or zero).
Hierarchical:
If a node is non-zero, its parent
must be non-zero
If a nodes parent is zero, the
node must be zero
Introduction
The Basics
Conclusions
Problem Statement
Goal:
Have sub-groups of sparse code
all be non-zero (or zero).
Hierarchical:
If a node is non-zero, its parent
must be non-zero
If a nodes parent is zero, the
node must be zero
Implementation:
Change the regularization
Enforce sparsity differently...
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Introduction
The Basics
Conclusions
Group Regularization
Updated objective function:
arg min
D,{j }
m h
X
j=1
kxj Dj k2
Introduction
The Basics
Group Regularization
Updated objective function:
arg min
D,{j }
m h
X
j=1
i
kxj Dj k2 + j
Conclusions
Introduction
The Basics
Group Regularization
Updated objective function:
arg min
D,{j }
m h
X
i
kxj Dj k2 + j
j=1
where
() =
X
gP
wg k|g k
Conclusions
Introduction
The Basics
Group Regularization
Updated objective function:
arg min
D,{j }
m h
X
i
kxj Dj k2 + j
j=1
where
() =
wg k|g k
gP
Conclusions
Introduction
The Basics
Group Regularization
Updated objective function:
arg min
D,{j }
m h
X
i
kxj Dj k2 + j
j=1
where
() =
wg k|g k
gP
Conclusions
Introduction
The Basics
Group Regularization
Updated objective function:
arg min
D,{j }
m h
X
i
kxj Dj k2 + j
j=1
where
() =
wg k|g k
gP
Conclusions
Introduction
The Basics
Conclusions
Other Examples
Introduction
The Basics
Outline
Conclusions
Conclusions
Introduction
The Basics
Outline
Conclusions
Conclusions
Introduction
Summary
The Basics
Conclusions
Introduction
The Basics
Conclusions
Summary
Two interesting
directions:
Introduction
The Basics
Conclusions
Summary
Two interesting
directions:
Increasing
speed of the
testing phase
Introduction
The Basics
Conclusions
Summary
Two interesting
directions:
Increasing
speed of the
testing phase
Optimizing
dictionary
structure