Lucas-Kanade

(wanglin193@hotmail.com)

1 Taylor expansion
() f(X) X=X
0

df
f ( X 0 + X ) f ( X 0 ) + X X =X0
dX
X n X=(x , x ,x ) df/dX
0 1 n

df df df
= , ,...
df
dX dx 0 dx1 dx n

f(X) m Jacobian
df 0 df 0
...
dx dxn
df 0
= ... ... ...
dX df m df m
dx ...
dxn
0
2

Chain rule
f t f t x,y ,f=f(x,y), x=x(t),y=y(t).
df/dt <,>
df f x f y f f x y
= + = , , ,
dt x t y t x y t t

L-K
p patch I(x) T(x)
E= [I (W ( X ; p)) T ( x)]
patch
2

W(X:p) p image warping I

(Affine transform W(X:p)(x,y)
x' x xp1 + yp3 + p5
W ( X ; p) = = +
y ' y xp2 + yp4 + p6
x
1 + p1 p3 p5
= y
p2 1 + p4 p6
1

6 3
4 2
 x' x xp1 yp2 + p3
y ' = y + xp + yp + p
2 1 4

x xs cos sin + p3
= +
y x sin + ys cos + p4

2 xy'' = yx ++ pp (Optical flow)

1

2
p 8

E E
2
I
E = [I (W ( X ; p + p )) T ( x)] I (W ( X ; p )) +
2
p T ( x)
patch patch p

Ep = 0

p p + ( I (W ( X ; p)) T ( x)) = 0
I
patch

I / p chain rule
I
=
I W
p W p
W Wy
= I , x
p p

I W / p
W x / p x 0 y 0 1 0
W / p =
y 0 1
.
y 0 x 0

I / p 12 26 16 patch
I
p = T ( x) I (W ( X ; p ))
p

p 6 (affine ) I / p
( p )
N N 6
Jp = E

J[N*6] I / p E [N*1] T-I

( J 6*6 )
 (
p = J T J )
1
J T E = dP * E

L-K

1 dP 6 N p
p=p+p
OpenCV
2
p 2dP T
Active Appearance Model warp
dP p (perturbation)CMU Cootes AAM
L-K Avatar

3 [1] H=J J Hessian

T
J Hessian
H H
p
2H (eigenvalue) 0
corner detection Good features to track

:
[1] Simon Baker and Iain Matthews. Lucas-Kanade 20 years on: A unifying framework.

[2] Jianbo Shi and Carlo Tomasi. Good features to track.

http://www.sciencenet.cn/u/wanglin193/