Neural networks

Computer vision - discrete convolution

COMPUTER VISION
Topics: parameter sharing
Each

Abst

Jarret
et
al.
2009
Math for my slides Computer vision.

feature map forms a 2D grid of features

can be computed with a discrete

convolution
H X ( ) of a kernel matrix kij which is
Bank
- FCSG
: its
therowsinput
of a flipped
filter bank
the hiddenLayer
weights matrix
Wij with
and columns

re

u
at
fe

ilter
r is a 3D array with n1 2D feature maps of size n2 n3 .
h component is denoted xijk , and each feature map
is
xi is the ith channel of input
oted xi . The output is also a 3D array, y composed
of
kij is the convolution kernelFigure 1. A example of fe
eature maps of size m2 m3 . A filter in the
bank
gj isfilter
a learned
scaling factorRabs N PA . An i
yj ismap
the hidden
layer
has size l1 l2 and connects input feature
xi to
through a non-linear fil
ut feature map yj . The module computes: (could have added a bias) contrast normalization an
!
yj = gj tanh(
kij xi )
(1)
layer with 4x4 down-s
m

s
ap

Figure 1. A example of feature extraction s

i

COMPUTER VISION
Topics: discrete convolution
convolution of an image x with a kernel k is computed
as follows:

The

(x * k)ij =
x
i+p,j+q kr-p,r-q
pq
Example:

~k = k
01 80
0.5 40
20
0.25 400

40

with rows and columns flipped

0

0.25

0.5

COMPUTER VISION
Topics: discrete convolution
convolution of an image x with a kernel k is computed
as follows:

The

(x * k)ij =
x
i+p,j+q kr-p,r-q
pq
Example:

1 x 0 + 0.5 x 80 + 0.25 x 20 + 0 x 40
01 80
0.5 40
20
0.25 400

40

0.25

0.5

45

COMPUTER VISION
Topics: discrete convolution
convolution of an image x with a kernel k is computed
as follows:

The

(x * k)ij =
x
i+p,j+q kr-p,r-q
pq
Example:

1 x 80 + 0.5 x 40 + 0.25 x 40 + 0 x 0
0

801 40
0.5

20

40
0.25 00

40

0.25

0.5

45

110

COMPUTER VISION
Topics: discrete convolution
convolution of an image x with a kernel k is computed
as follows:

The

(x * k)ij =
x
i+p,j+q kr-p,r-q
pq
Example:
0

80

40

201 40
0.5 0
0
0 0 40
0.25

1 x 20 + 0.5 x 40 + 0.25 x 0 + 0 x 0

0.25

0.5

45
40

110

COMPUTER VISION
Topics: discrete convolution
convolution of an image x with a kernel k is computed
as follows:

The

(x * k)ij =
x
i+p,j+q kr-p,r-q
pq
Example:
0

80

40

20

401

00.5

0 400
0.25

1 x 40 + 0.5 x 0 + 0.25 x 0 + 0 x 40

0.25

0.5

45

110

40

40

COMPUTER VISION
Topics: discrete convolution
Pre-activations

from channel xi into feature map yj can be

computed by:

getting the convolution kernel where kij =Wij from the connection matrix Wij

applying the convolution xi * kij

This

is equivalent to computing the discrete correlation

of xi with Wij

COMPUTER VISION

Example
Example
Topics: discrete convolution

0%
~
Simple illustration:

Calcul%dune%couche%%simple%cell%%
x
k
where
W
=W
i
*
ij
ij
ij
Calcul%dune%couche%%simple%cell%%

premire%tape%:%calcul%de%la%convolu7on%% 0.5%
premire%tape%:%calcul%de%la%convolu7on%%

0.5%

0.5% 0% 0% 0%

255%
0%
0%

0%

0% 0.5% 0%
255%
0% 0.5% 0%
255% 0%0%
0%
0%
0%
0%
255%
0%
0%
255%
0%
0%
0%
255%
0%
0%
255%
0%
0%
0%
255%
0%
0%
255%
0%
0%
0%
0%

0%

0% 0% 0%

255%

0%
0%
0%

%%%%%
X
x
%%%%%
%
Xi couche)dentre)
%
couche)dentre)

0%
0%
0%

0.5%

connexions%
connexions%
vers%les%neurones%
0.5% vers%les%neurones%
0%
cachs%
0%
cachs%

W
W

0%
0%

0%
0.5%

0%
0%
0%
255%

128% 128%
128%
0%

0%

128% 128%
128% 128%
0%
0%
255%
0%
255%
0%
0%
255%
0%
0%
0%
0%
0%

0%

128%

0%
0%

0%
0%

X
W
%%%%%
x
k
i * ij
X couche))simple)cell))
W%%%%% %

%
couche))simple)cell))

COMPUTER VISION
Example
Topics: discrete convolution
With

a non-linearity, we get a detector of a feature at any

Calcul%dune%couche%%simple%cell%%
position in the image
premire%tape%:%calcul%de%la%convolu7on%%

deuxime%tape%:%calcul%de%la%nonnlinarit%(%ex.:%Logis6c(%(xn200)/50%)%)%

0%

0%

255%

0%

0%

0%

0%

255%

0%

0%

0%

0%

255%

0%

0%

0%

255%

0%

0%

0%

255%

0%

0%

0%

0%

x
%%%%%
Xi

%
couche)dentre)

0.02%
0.19%
0%
128% 0.19%
128% 0.02%
0%
0.02%
0.19%
0%
128% 0.19%
128% 0.02%
0%
0.02%
0.75%
0.02%
0%
255% 0.02%
0%
0%
0.75%
0.02%
0.02%
255% 0.02%
0%
0%
0%

sigm(0.02
x
k
-4)
i
*
ij
Logis6c(%(%%%%%%%%%%%%%n%200%)%/%50%)%
X W%%%%%
%
couche))simple)cell))

10

COMPUTER VISION
Topics: discrete convolution
Example
Can

0%

use zero padding to allow going

Convolu7on%avec%zero,padding%%
0% 0.5%
0%

0%

0%

0%

0%

0%

0% 0.5% 0%

0% 0%

255%

0%

0%

0%

0%

0%

0%

255%

0%

0%

0%

0%

0%

0%

255%

0%

0%

0%

0%

0%

255%

0%

0%

0%

0%

0%

255%

0%

0%

0%

0%

0%

0%

0%

0% %%%%% 0%

0%

0%

0%

%
couche)dentre)
i

0%

0.5%

connexions%
vers%les%neurones%
over the borders
(*)
0.5%
0%
cachs%

W
0%
128%

0%

0%

0%

0%

0%

0%

128%

128%

0%

0%

0%

0%

128%

128%

0%

0%

0%

0%

255%

0%

0%

0%

0%

255%

0%

0%

0%

0%

128%

0%

0%

0%

0%

0%

%%%%%
xi *% kij
couche))simple)cell))

11