Professional Documents
Culture Documents
07)
Shahram Ebadollahi
Description
Recognition
High-level Image
Representation
Image
4/15/2008 Understanding 2
Lecture Outline
1 Image Description
1 Shape Descriptors
1 Texture & Texture Descriptors
1 SIFT
1 Motion Descriptors
1 Color Descriptors
4/15/2008 3
Shape Description
1 Shape Represented by its Boundary
1 Shape Numbers,
1 Fourier Descriptors,
1 Statistical Moments
4/15/2008 4
Boundary Representation:
(Freeman) Chain Code
Boundary representation = 0766666453321212
4/15/2008 5
Chain Code: example
4/15/2008 7
Boundary descriptor – Fourier
s (k ) = x(k ) + jy (k ) k = 0,1,2, 1 , K − 1
K −1
a (u ) = ∑ s (k )e − j 2πuk / K u = 0,1,2, 1 , K − 1 Fourier Descriptor
k =0
K −1
1
s (k ) =
K
∑ a
u =0
(u ) e j 2πuk / K
k = 0,1,2, 1 , K − 1
4/15/2008 8
Boundary Reconstruction using
Fourier Descriptors
100% 50% 10% 5%
2868
descriptors
Only 8
descriptors
4/15/2008 9
Boundary Representation: Signatures
• Represent 2-D boundary shape using 1-D signature signal
4/15/2008 10
Boundary Representation: Signatures
4/15/2008 11
Boundary Description using
Statistical Moments
A−1
µ n (v) = ∑ (vi − m) n p(vi ) n-th moment of v
i =0
A−1
m = ∑ vi p (vi )
i =1
4/15/2008 12
Region Descriptors - Simple
1 Area
1 Perimeter
1 Compactness (perimeter)2/Area
1
r
Circularity Ratio 2
1 Mean/Median intensity r b
πr
1 Max/Min intensity a=2b
1 Normalized area
C : 4π 5π 16
A
Rc = 2 4 ≈ 0.8 π ≈ 0.78
P / 4π Rc : 1 5 4
Area of circle with same
perimeter as the shape
4/15/2008 13
4/15/2008 14
Topological Region Descriptors
• Topological properties: Properties of image preserved under
rubber-sheet distortions
V −Q + F = C − H = E
H=1, C=1, E=0 H=2, C=1, E=-1
4/15/2008 15
Geometric Moment Invariants
4/15/2008 16
Central Moments
M −1 N −1
µ pq = ∑∑ ( x − x0 ) p ( y − y0 ) q f ( x, y ) µ 00 = m00
x =0 y =0
µ10 = µ 01 = 0
[Translation invariance]
µ 02 , µ 20 Variance about the centroid
µ11 covariance
4/15/2008
η pq = µ pq /( µ00 ) ( p + q + 2 ) / 2 17
Moment Invariants
(translation, scale, mirroring, rotation)
φ1 = η 20 + η 02
φ2 = (η 20 − η 02 ) 2 + 4η112
φ3 = (η30 − η12 ) 2 + (η 21 − η 03 ) 2
φ4 = (η30 + η12 ) 2 + (η 21 + η 03 ) 2
ϕ1 = η 20 + η
φ5 = 1
φ6 = 1
φ7 = 1
4/15/2008 18
Affine Transform & Affine Moment Invariants
m m−r
x' = ∑∑ ark x r y k
x' = Tx ( x, y ) r =0 k =0
m m−r
y ' = Ty ( x, y ) y ' = ∑∑ brk x r y k
r =0 k =0
θ
( I1 + I 2 ) / m00
2
spreadness 2µ11
θ = 0.5 tan −1 ( )
µ 20 − µ 02
( I 2 − I1 ) /( I1 + I 2 ) elongation
a = 2( I1 / µ 00 )1/ 2 b = 2( I 2 / µ 00 )1/ 2
4/15/2008 20
Texture - Definition
4/15/2008 21
Texture – Quantification Methods
4/15/2008 22
Statistical Texture Analysis
1st order statistics
image f → hf histogram
L −1
Variance:
∑ (i − µ )
i =0
2
h(i ) : measure of intensity contrast
L −1
Skewness: ∑ (i
i =0
− µ ) 3
h(i )
L −1
Entropy: − ∑ h(i ) log h(i ) Measure of variability of intensity
4/15/2008 i =0 23
4/15/2008 24
Statistical Texture Analysis
1st order statistics
image
histogram
statistics
4/15/2008 25
Statistical Texture Analysis
2nd order statistics: Co-occurrence
f (m2 , n2 ) = j
d
θ P( d ,θ ) (i, j ) ≈ Pr[ f (m1 , n1 ) = i, f (m2 , n2 ) = j ]
f (m1 , n1 ) = i
1 2 j L
1
1
Joint gray-level histogram of pairs of
2
pixels P( d ,θ ) ( i , j )
1 2D histogram
i
4/15/2008 26
Statistical Texture Analysis
2nd order statistics: Co-occurrence
P( d ,θ =0° ) (i, j ) =| {((k , l ), (m, n)) ∈ ( M × N ) × ( M × N ) :
k − m = 0, | l − n | = d , f (k , l ) = i, f (m, n) = j} |
P( d ,θ = 45° ) (i, j ) =| {((k , l ), (m, n)) ∈ ( M × N ) × ( M × N ) :
(k − m = d , l − n = −d ) ∨ (k = m = −d , l − n = d ), f (k , l ) = i, f (m, n) = j} |
P( d ,θ =90° ) (i, j )
90°
P( d ,θ =135° ) (i, j )
135° 45°
k
4/15/2008 28
Statistical Texture Analysis
2nd order statistics: Co-occurrence (statistics)
L L
i =1 j =1
(measure of image homogeneity)
L L
− ∑∑ P( d ,θ ) (i, j ) logP( d ,θ ) (i, j )
Entropy: i =1 j =1
L L
∑∑ λ κ
i − j P( d ,θ ) (i , j )
Contrast: i =1 j =1
(measure of local variations) L L
∑∑ [ijP
i =1 j =1
( d ,θ ) (i, j )] − µ x µ y
Correlation:
(measure of image linearity) σ xσ y
L L L L
4/15/2008 µ x = ∑ i ∑ P( d ,θ ) (i, j ) σ x = ∑ (i − µ x ) 2
∑P ( d ,θ ) (i, j ) 29
i =1 j =1 i =1 j =1
4/15/2008 30
Statistical Texture Analysis
2nd order statistics: Difference Statistics
P( d ,θ ) (k ) = ∑P ( d ,θ )
i , j∈{1,1, L}
(i, j ) is a subset of co-occurrence matrix
|i − j | = k
L −1
k =0
L −1
Mean: ∑ kP
k =0
( d ,θ ) (k )
L −1
Entropy: − ∑ P( d ,θ ) (k ) log P( d ,θ ) (k )
k =0
L −1
Contrast: ∑ P(d ,θ ) (k )
k 2
k =0
4/15/2008 31
Statistical Texture Analysis
2nd order statistics: Autocorrelation
M − pN −q
MN ∑ ∑ f ( k , l ) f ( k + p, l + q )
C ff ( p, q ) = k =1 l =1
( M − p )( N − q ) M N
∑∑
k =1 l =1
f 2 (k , l )
4/15/2008 32
Statistical Texture Analysis
2nd order statistics: Fourier Power Spectrum
f ( x, y ) ↔ F (u , v)
Note:
Power Spectrum P(u, v) =| F (u , v) |2
C ff = F −1{| F (u, v) |2 }
v
π Indicator for size of
P ( r ) = 2∑ P ( r , θ ) dominant texture element
u or texture coarseness
θ =0
L/2
v
P(θ ) = ∑ P(r , θ )
Indicator for the
r =0
directionality of the texture
u
4/15/2008 33
Law’s Texture Energy Measures
L3 = [1,2,1] E3 = [−1,0,1] S3 = [−1,2,−1]
L5 = [1,4,6,4,1]
E5 = [−1,−2,0,2,1]
S5 = [−1,0,2,0,−1]
R5 = [1,−4,6,−4,1]
W5 = [−1,2,0,−2,−1]
−1 0 0 − 1
2
− 4 0 0 − 4
8
•Convolute different Law’s
L5 × S5 = − 6
T
0 12 0 − 6 masks with image
− 4 0 8 0 − 4 • Compute energy statistics
4/15/2008
− 1 0 2 0 − 1 34
4/15/2008 35
4/15/2008 36
4/15/2008 37
4/15/2008 38
4/15/2008 39
Motion – object
Difference image:
d (i, j ) = 0 if | f1 (i, j ) − f 2 (i, j ) | ≤ ε object
motion
1 otherwise
n
d cum (i, j ) = ∑ ak | f1 (i, j ) − f k (i, j ) |
k =1 Cumulative difference image
4/15/2008 40
Motion Field
• A velocity vector is assigned to each pixel in the image
• Velocities due to relative motion between camera and the 3D
scene
• Image change due to motion during a time interval dt
• Velocity field that represents 3-dimensional motion of object
points across 2-dimensional image
Motion field
4/15/2008 41
Optical flow
• Motion of brightness patterns in image sequence
∂f ∂f ∂f
f ( x + dx, y + dy, t + dt ) = f ( x, y, t ) + dx + dy + dt + O(∂ 2 )
∂x ∂y ∂t
Gray-level difference at same
= f ( x, y, t ) + f x dx + f y dy + f t dt + O(∂ )
2 location over time is
equivalent to product of spatial
gray-level difference and
dx dy velocity
f ( x + dx, y + dy, t + dt ) = f ( x, y, t ) ⇒ − f t = f x + fy
dt dt
− f t = f x u + f y v = ∇f .c
dx dy
c=( , ) = (u, v)
dt dt
4/15/2008 known unknown 42
Optical Flow Constraints
− f t = f xu + f y v
4/15/2008 43
Optical flow != Motion Field
MF ≠ 0 MF = 0
OF = 0 OF ≠ 0
4/15/2008 44
Which descriptors?
Image Feature Evaluation
1. Prototype Performance
1 Classify (Segment) image using
different features
1 Evaluate which feature is optimal
(minimum classification error)
2. Figure of Merit
1 Establish functional distance
measurements between set of
image features (large distance 2
low classification error)
1 Bhattacharyya distance
4/15/2008 45