You are on page 1of 108

Pixel-Based Processing

ECE 847: Stan Birchfield


Digital Image Processing Clemson University
Challenge

original image detect objects, classify fruit,


find banana stem
Steps:
1. binarize / threshold
2. clean up binary image
3. find regions corresponding to foreground objects
4. compute properties of regions
5. classify regions based on properties

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Outline
• Pixel-based image transformations
(thresholding, …)

• Removing noise from binary image


(morphological operators)

• Labeling regions
(floodfill, connected components)

• Computing distance in an image


(chamfer distance, …)

• Region properties
(area and boundary properties)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


What is a digital image?

75 81 83 96 94
62 74 76 87 100
74 76 80 89 93
53 67 86 90 105
60 77 90 99 115

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Accessing pixels in an image
Timing Test
(640 x 480 image,
get/set each pixel
C / C++ using 2.8 GHz P4)
• Method 1: 1D array
val = p[ y * width + x]; 6.6 ms
• Method 2: 2D array
val = p[y][x]; 5.6 ms
• Method 3: pointers
val = *p++; 0.5 ms
(even faster with SIMD operations)
Matlab
• Method 1: 2D array
21.4 ms
val = im(y,x);
• Method 2: parallelize
im = im + constant; 2.5 ms

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Outline
• Pixel-based image transformations
(thresholding, …)

• Removing noise from binary image


(morphological operators)

• Labeling regions
(floodfill, connected components)

• Computing distance in an image


(chamfer distance, …)

• Region properties
(area and boundary properties)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Types of image
transformations
• Graylevel transforms
I’(x,y)  f( I(x,y) )
(arithmetic, logical, thresholding, histogram
equalization, …)
• Geometric transforms
I’(x,y)  f( I(x’,y’) )
(flip, flop, rotate, scale, …)
• Area-based transforms
I’(x,y)  f( I(x,y), I(x+1,y+1), … )
(morphological operators, convolution)
• Global transforms
I’(x,y)  f( I(x’,y’), x’,y’ )
A
(Fourier transform, wavelet transform)
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Graylevel transforms

identity inverse contrast threshold


enhancement

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Geometric transforms

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Histograms

image histogram

Throws away all spatial information!


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Histogram equalization

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Running sum
Step 1: Initialize 1D array
int hist[256]

...

Step 2: Compute running sum


sum[0] = 0
for g = 1 to 255,
sum[g] = sum[g-1] + hist[g]

Example:
[ 6 3 9 7 4 2]  [6 9 18 25 29 31]

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Histogram equalization
pdf

Steps:
1. Compute normalized histogram (PDF) equal area
int hist[256] = 0…0
float norm[256]
for x, hist[ I(x) ]++
for g = 0 to 255,
norm[g] = hist[g] / npixels
cdf
2. Compute cumulative histogram (CDF)

new gray level


cum[0] = norm[0]
for g = 1 to 255, equal spacing
cum[g] = cum[g-1] + norm[g]

3. Transform
for x, I(x) = 255*cum[ I(x) ]

running sum old gray level

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Histogram equalization *
PDF desired
p
PDF
p’
equal area equal area

a a’ a’+d new gray level k’


CDF
d{ By definition,
new gray level k’

q
d{ q 1 ( a ' d )
a’
equal  1
q ( a ')
p( x)dx
spacing
 q(q 1 (a'd ))  q(q 1 (a' ))

old gray level k


d

where p is the PDF and q is the CDF


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Histogram equalization *
PDF desired
p PDF
equal area
p’
1/(L-1)

a b L-1 a’ a’+d L-1


old gray level k new gray level k’
scaled CDF
b
 p (k )dk  q (b)  q(a )
L-1
q·(L-1)
d{
new gray level k’

a
a’  a'd  a ' /( L  1)
 d /( L  1)
a 'd
 p' (k )dk '
a'
L-1
old gray level k
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Histogram equalization *
PDF desired
p PDF
equal area
p’
1/(L-1)

a1 b1 a2 b2 L-1 a1’ a1’+d a2’ a2’+d L-1


new gray level k’
scaled CDF
d
b
 p (k )dk  q (b)  q(a )
L-1
{
a2’ q·(L-1)
d{
new gray level k’

a
a1’  a'd  a ' /( L  1)
 d /( L  1)
a 'd
 p' (k )dk '
a'
L-1
old gray level k
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Integral image
• Integral image is a 2D running sum
• S(x,y) = SS I(x,y)
• To compute,
S(x,y) = I(x,y)
- S(x-1,y-1) + S(x-1,y) + S(x,y-1)
• To use,
V(l,t,r,b) = S(l,t) + S(r,b) - S(l,b) - S(r,t)
Returns sum of values inside rectangle
• Note: Sum of values in any rectangle
can be computed in constant time!

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Thresholding

Valley separates
light from dark

How to find it?

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


A simple thresholding
algorithm

Repeat until convergence:


T  ½ (m1 + m2)
T. Ridler and S. Calvard, Picture Thresholding Using an Iterative Selection Method, IEEE
Transactions on Systems, Man and Cybernetics, 8(8):630--632, August 1978.
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Ridler-Calvard algorithm

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Otsu’s method

• Threshold t splits image into two groups


• Calculate within-group variance of each
group
• Search over all t to minimize total within-
group variance; moment calculations
make search efficient
 w2 (t )  q1 (t ) 12 (t )  q2 (t ) 22 (t )

percentage of pixels variance of pixels


in first group in first group

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Otsu’s method

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Local Entropy
Thresholding (LET) *

• Compute co-occurrence matrix


• Threshold s divides matrix into
four quadrants
• Pick s that maximizes
HBB(s)+HFF(s)
s s
H BB ( s)   p BB
ij log p BB
ij
i 0 j 0

• Uses spatial information


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Otsu vs. LET *

image histogram Otsu LET

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Adaptive thresholding

background image histogram

from Gonzalez and Woods, p. 597


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Double thresholding
graylevel
(hysteresis)
threshold too high:
misses part of object

object 1 object 2

threshold too low:


captures noise
noise
pixel
Algorithm:
1. Threshold with high value
2. Threshold with low value, retaining only the
pixels that are contiguous with existing ones
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Double thresholding example

1. Use high threshold


low threshold high threshold
to get seed pixel

3. Repeat

2. Floodfill
using seed pixel
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Double thresholding example

image low threshold retains some background

high threshold removes some foreground combined using floodfill on low threshold
with seeds from high threshold
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Double thresholding

We will explain Floodfill later

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Outline
• Pixel-based image transformations
(thresholding, …)

• Removing noise from binary image


(morphological operators)

• Labeling regions
(floodfill, connected components)

• Computing distance in an image


(chamfer distance, …)

• Region properties
(area and boundary properties)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Binary image as a set

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Set operations

Fundamental
operators

Other
operators

DeMorgan’s
Laws

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Minkowski operators

Minkowski
addition:

Minkowski
subtraction:

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Minkowski addition

Add each element of B


to each element of A1
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Minkowski subtraction

Intersection of translating
A2 to every point in B

Two examples:

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Minkowski addition:
“Center-out” approach

(“center-out” is more intuitive)


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Minkowski subtraction:
“Center-out” approach

(“center-out” is more intuitive)


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Morphological operations

• morphology – study of form or shape


• Mathematical morphology – grew out
of research at Fontainebleau in 1964
• We will deal only with binary images
• Two fundamental operations:
Dilation and Erosion
(same as Mink. add.)
(same as Mink. sub.
B is the structuring element (SE) after reflection)
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Dilation

• Algorithm (center-out):
– Flip B horizontally and vertically
(but often B is symmetric, so nothing is changed)
– Move B around
– Anywhere they intersect, turn center pixel on
• Looks like nonlinear convolution

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Erosion

• Algorithm (center-out):
– Move B around Duality b/w dilation
– If there is not complete overlap, and erosion:
turn center pixel off

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Common structuring
elements
• Usually SE is either B4 or B8
• Greatly simplifies algorithm
(and erosion is same as Min. sub.)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Dilation example

Let B be 3x3 with all 1s. B


for y
for x
if ( in(x-1, y)
|| in(x-1, y-1)
|| in(x-1, y+1)
...
|| in(x+1, y+1)
)
out(x, y) = 1
else
out(x, y) = 0 A

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Erosion example

Let B be 3x3 with all 1s. B


for y
for x
if ( in(x-1, y)
&& in(x-1, y-1)
&& in(x-1, y+1)
...
&& in(x+1, y+1)
)
out(x, y) = 1
else
out(x, y) = 0
A

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Why does dilation require flipping
the structuring element?
Simple example:

A B
Without flipping With flipping

 

Result is flipped! Fixed!


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Closing and Opening
• Dilation fills gaps. Erosion removes noise
• Combine them:
– First dilate, then erode (closing)

– First erode, then dilate (opening)

• Closing/opening retain object size


• Repeated applications do nothing

• Duality between closing and opening

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Closing and Opening
Closing: dilate, then erode

Opening: erode, then dilate

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Morphology Example

thresholded difference open erode


dilate
close
open
then close
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Accessing out-of-bounds
pixels
• If kernel is near boundary of image, some pixels of kernel
will be out of bounds (OOB)
• What to do?
– Do not use OOB pixels (change kernel size near boundary)
– Do not compute output for pixels near boundary
– Extend image function:
• zero pad – OOB pixels have value of zero
• replicate – OOB pixels have value of closest pixel
• reflect – pixel values are extended by mirror reflection at border
• wrap – pixel values are extended by repeating
• extrapolate – pixel values are extended by extrapolating function
near boundary
• No solution is best
• Problem also occurs with convolution

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Outline
• Pixel-based image transformations
(thresholding, …)

• Removing noise from binary image


(morphological operators)

• Labeling regions
(floodfill, connected components)

• Computing distance in an image


(chamfer distance, …)

• Region properties
(area and boundary properties)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Binary images

• Each pixel is ON or OFF


• We will say that foreground pixels are ON,
background pixels are OFF
• In computer, each pixel has value of 0 or
1.
• We adopt convention that 0=OFF, 1=ON.
• Usually 0=black and 1=white, but
sometimes display is reversed

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Neighbors

4-neighbors 8-neighbors diagonal neighbors

N4 N8 ND

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Adjacency

• 4-adjacency: q  N4 ( p) p q

• 8-adjacency: q  N8 ( p) p
q

• m-adjacency: q  N4 ( p) or
q  N D ( p) and N4 ( p)  N4 (q)  

Example:

48m
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Adjacency
Use different connectedness for
foreground and background to avoid
inconsistency

foreground: D8

background: D4

dumbbell

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Connectivity, Regions and
Boundaries
• path from p to q is sequence of adjacent
pixels
• p and q are connected in subset S if path
exists between them containing only
pixels in S
• connected component of p is set of pixels
in S that are connected to p
• S is a region if it is a connected set
• boundary of S is the set of pixel in S that
have one or more neighbors not in S
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Boundary definitions

binary outer outer outer


image region hole complete
boundary boundary boundary

inner inner inner


region hole complete
boundary boundary boundary
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Floodfill
• Fill a region with a new color
– Start with a seed point
– All pixels are colored if they have the same
color as the seed and are connected to the
seed via pixels that have the same color as the
seed

seed pixel

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Floodfill *
Repeated applications of dilation can fill a region

image A structuring complement Ac


element B
... but terribly inefficient S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Floodfill
Efficient implementation uses a stack called the frontier.
Function image =
Floodfill(seed_point,
image,
new_color)
old_color = image(seed_point)
if (old_color == new_color) return
frontier.Push(seed_point)
image(seed_point) = newcolor
while (NOT frontier.IsEmpty())
q = frontier.Pop();
for each neighbor r of q,
if image(r) == old_color,
frontier.Push(r)
image(r) = new_color
return image
frontier:
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Floodfill

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Floodfill

This code performs the same logic,


but does not change the pixels of the input image
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Revisit:
Double thresholding

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Connected components

• Recall that connected component of


p is set of pixels in S that are
connected to p
• Can solve the problem by repeated
applications of Floodfill – assigning a
new label to each new region
• Slightly more efficient
implementation involves two passes
through the image (union-find)
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Connected components using
floodfill

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Classical union-find connected
component labeling algorithm
scan left-to-right, top-to-bottom
U
L P
4-neighbor I(P) == I(U) ?
algorithm: I(P) == I(L) ? YES NO

YES C(P)=C(U) and C(P)=C(L)


set Equiv(U,L)

NO C(P)=C(U) C(P)=n++

4-neighbor 8-neighbor
mask: mask:
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Classical connected
components: first pass
1 1 1 1 1 0 0 0 0 0
0 0 0 0 1 1 1 1 1 0
1 1 1 1 1 2 2 2 2 0
0 0 0 1 0 3 3 3 2 4
0 1 1 1 0 3 5 5 2 4
image components

0 1 2 3 4 5

equivalence table
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Classical connected
components: traverse table
1 1 1 1 1 0 0 0 0 0
0 0 0 0 1 1 1 1 1 0
1 1 1 1 1 2 2 2 2 0
0 0 0 1 0 3 3 3 2 4
0 1 1 1 0 3 5 5 2 4
image components

0 1 2 3 4 5

equivalence table
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Classical connected
components: second pass
1 1 1 1 1 0 0 0 0 0
0 0 0 0 1 1 1 1 1 0
1 1 1 1 1 0 0 0 0 0
0 0 0 1 0 3 3 3 0 4
0 1 1 1 0 3 0 0 0 4
image components

0 1 2 3 4 5

equivalence table
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Another example

from http://www.sciencedirect.com/science/article/pii/S0031320308004573
Connected components
Function labels = ConnectedComponents(image)

// first pass
for y=0 to height-1,
for x=0 to width-1,
pix = image(x,y)
if pix == image(x-1,y) && pix == image(x,y-1)
labels(x,y) = labels(x,y-1)
SetEquivalence(labels(x-1,y), labels(x,y-1))
else if pix == image(x-1,y)
labels(x,y) = labels(x-1,y)
else if pix == image(x,y-1)
labels(x,y) = labels(x,y-1)
else
labels(x,y) = next_label++

// second pass
for y=0 to height-1,
for x=0 to width-1,
labels(x,y) = GetEquivalentLabel( labels(x,y) )
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Connected components
Function SetEquivalence(a, b)
aa = GetEquivalentLabel(a)
bb = GetEquivalentLabel(b)
if aa > bb
equiv[aa] = bb
else if aa < bb
equiv[bb] = aa

Function b = GetEquivalentLabel(a)
if a == equiv[a] return a
else
equiv[a] = GetEquivalentLabel(equiv[a])
return equiv[a]

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Connected components

quantized image components

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Outline
• Pixel-based image transformations
(thresholding, …)

• Removing noise from binary image


(morphological operators)

• Labeling regions
(floodfill, connected components)

• Computing distance in an image


(chamfer distance, …)

• Region properties
(area and boundary properties)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Perimeter
• Perimeter of binary region:
– Dilate, then subtract
– Erode, then subtract
– (Which is better?)
• This only gives the pixels as binary
image
• To get ordered list of contiguous
pixels, use Wall Follow algorithm

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Wall following
Algorithm:
1. Start at ON pixel
2. Move in any direction until end of
region reached
3. Turn right
4. Repeat:
1. If pixel to the left is ON,
Turn left and move forward
2. Else if pixel in front is OFF,
Turn right
3. Else
move forward
- This traverses clockwise, using 4-connectedness
(other versions are possible)
- Can be used to compute perimeter
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Distance functions

• D is a distance function (metric) if

• Common distance functions:

shape of each?
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Other distances

• Mahalanobis
(weight differently along each axis)

• Hausdorff
(distance b/w point sets)
A and B are subsets of S

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Length along path # of isothetic
# of diagonal
moves
moves

Freeman formula Pythagorean theorem Kimura’s method


c=0 c=1 c=½
(overestimates) (underestimates) (good compromise)
L  2 Nd  No L  [ N d2  ( N d  N o ) 2 ]1/ 2 L  [ N d2  ( N d  N o / 2) 2 ]1/ 2  N o / 2

3 methods compared: Freeman:


L = 16.5
Example:

No=8 Pythagorean:
L = 15.2

Kimura:
L = 15.7
Nd=6
circle with radius = 10 True arc length:
* isothetic (blue) L = 10*p/2 = 15.7
Nd=6
* diagonal (orange) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Chamfering
• How to approximate “correct” distance
(Euclidean)?
• Along a path,
– Let m be number of isothetic moves (horizontal or vertical)
– Let n be number of diagonal moves
• Define chamfer distance between p and q as
dab = min {am+bn} over all possible paths
• Chamfer distance is metric if a, b > 0
• If b > 2a, then diagonal moves are ignored
• Special cases:
– a=1, b=infinity (Manhattan)
– a=1, b=1 (chessboard)
– a=1, b=sqrt(2) (another reasonable choice)
– a=1, b=1.351 (best approximation to Euclidean)
– a=3, b=4 (best integer approximation, scaled by factor of 3)
[G. Borgefors, Distance transformations in digital images, CVGIP, 1986]
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Chamfering (cont.)
• Chamfering is process of computing distance to
each foreground pixel
• Two-pass algorithm
– Traverse top-to-bottom, left-to-right
– Traverse bottom-to-top, right-to-left
In each case, compute minimum distance foreground pixels

L: R: L: R:

4-connectivity 8-connectivity
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Chamfering (cont.)
• Algorithm (for D4 using Manhattan distance):
Function labels = Chamfer(image)
// first pass
for y=0 to height-1,
for x=0 to width-1,
d(x,y) = image(x,y) ? 0
: min(infinity, 1+d(x-1,y), 1+d(x,y-1))
// second pass
for y=height-1 to 0 step -1,
for x=width-1 to 0 step -1,
d(x,y) = image(x,y) ? 0
: min(d(x,y), 1+d(x+1,y), 1+d(x,y+1))

Note:

“? :” means “if-else”

(C conditional
expression operator)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Uses of chamfering
• Find center of biggest
centroid
part of shape (need to
largest chamfer
compute signed distance) value

• Shape matching

http://www.gavrila.net/Research/Chamfer_System/chamfer_system.html

• Useful for watershed thresholding


(we will see this later)
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Outline
• Pixel-based image transformations
(thresholding, …)

• Removing noise from binary image


(morphological operators)

• Labeling regions
(floodfill, connected components)

• Computing distance in an image


(chamfer distance, …)

• Region properties
(area and boundary properties)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Descriptors

• Region

}
– Euler number
– convex hull
– skeleton We will focus
– moments
– compactness
on regions
– eccentricity (elongatedness)
• Boundary
– chain code
– signature
– r-q plot
– s-y plot
– boundary segments
– shape number
– Fourier descriptors

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Euler number

E = C- H
components holes

1-1=0 1-2=-1 1-0=1 2-0=2


Measure of topology
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Euler number

• Euler number computed as (N1-N2-2N3)4,


where
– N1 is the number of 2x2 patterns with 1 in
either corner, 0 elsewhere (4 possible patterns)
– N2 is the number of 2x2 patterns with 0 in
either corner , 0 elsewhere (4 possible
patterns)
– N3 is the number of 2x2 checkerboard patterns
(2 possible patterns)
Explained in W.K. Pratt, Digital Image
Processing, 1991

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Convex hull
• Convex – any line connecting two points
in R is also in R

convex not convex

• Convex hull – the minimal convex set


containing R

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Skeletons
• Point on skeleton is should skeleton
touch boundary?
equidistant to >= 2
boundary points
(without crossing)

• Or, equivalently,
– where wavefronts (grassfires) meet (shock graph)
– centers of maximal balls

– Blum’s Medial Axis Transform (MAT)


• Skeleton is highly sensitive to noise

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Skeleton examples

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Skeletonization by thinning

• First compute MAT


• Then delete points such that
– do not remove endpoints
– do not break connectivity
– do not excessively erode region

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Moments in 1D
• The pth moment of a function f:

• The pth central moment:

centroid
• What are ?
(area, 0, variance, skew, kurtosis)

m2 is also moment of inertia about axis passing through centroid


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Moments in 2D
• The pqth moment of a 2D region:
can be
• grayscale or
• binary (more typical)

• The pqth central moment:

(area)
(variance in x)
(variance in y)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Moments in 2D (cont.)
• The pqth normalized central moment:

• Hu moments (derived from these) are invariant to


– translation
– rotation
– scaling
– skew (at least one of them is)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Moments example
• Simple 5x5 binary image:

m00 = 12.0 m00 = 12.0


m10 = 23.0 m10 = 0.0
m01 = 21.0 m01 = 0.0
m11 = 37.0 m11 = -3.3
m20 = 65.0 m20 = 20.9
m02 = 63.0 m02 = 26.3
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Computing moments

One pass through image


to compute
regular moments

Central moments
(and normalized / Hu)
are computed without
touching the pixels again!

(Typically, f is a binarized image containing a single non-zero region)


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Projections

• Project image onto x, y, or diagonal


• Projection offers compact
representation of image
• First-order moments of image are
equal to first-order moments of
projection
• Second-order moments require
diagonal projection

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Compactness
Related to eccentricity (elongatedness)
• Define compactness as (area = # of pixels,
perimeter = length along
boundary)

compactness = 1 compactness < 1

(Note: discretization effects of computing perimeter are severe)

Alternative: Use
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Eccentricity
• Approximate region with ellipse
• eccentricity = a / a =
• Formula given by PCA

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Principal components
analysis (PCA)
Given n points in d dimensional space

Goals:
1. fit ellipsoid to data
2. reduce dimensionality

Ellipsoid specified by
• center
(given by mean of data)
• orientation of axes
(given by eigenvectors of covariance matrix of
data)
• length of axes
(given by square root of eigenvalues of covariance
matrix of data)
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
PCA (cont.)
• n points in d dimensional • eigenvalues and
space eigenvectors

• matrix form
• mean

• covariance • factorize covariance


matrix

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


PCA (cont.)

• linear transformation • reconstruct x

• but elements of y are


• data uncorrelated! decreasingly important

• vectors ei form an • project to k-dimensional


orthonormal basis for space
space

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


PCA (cont.)

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


PCA example
• Simple 5x5 binary image:
m = (2.0, 2.0)

C = (1/9) * [ 12 -10]
[-10 12]
e1 e2 l1
C = PLPT = [-0.7 0.7] [2.4 0 ] [-0.7 0.7]
[ 0.7 0.7] [ 0 0.2] [ 0.7 0.7]
l2
major axis = e1√l1 = [1.1 -1.1]T

minor axis = e2 √l 2 = [-0.3 -0.3] T

S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847


Best fitting ellipse
The elliptical binary region in image

bounded by the ellipse

has the following second-order moments

To be an ellipse, (which is always


  true for a
covariance matrix)
Note:
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Eccentricity and Direction
• Covariance matrix
of region

• Eigenvalues
solve det(C-lI)=0)

• Eccentricity
ranges from
0 (circle) to 1 (line)

Hint: Use atan2(y,x),


• Direction passing numerator
(clockwise angle from and denominator
positive x-axis) as separate arguments

• Eigenvectors
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Eigenfaces
mean face

eigenvectors
N x N image is a point in an N2 dimensional space
N2 x N2 Covariance matrix captures relationships
between face images in ensemble
Turk and Pentland, 1991
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Fruit classification

(high threshold) original image (low threshold)


Note: double threshold
and connected components
can be computed simultaneously:
• scan high threshold image
• if pixel is 1, then floodfill
using low threshold image
(connected components) (double threshold)

mij mij
(moments)

(classification) (banana stem)


S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847
Sample results

fruit1.pgm fruit2.pgm
S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

You might also like