Segmentation

Chapter Overview: Image Segmentation !
Gray-level thresholding! Supervised vs. unsupervised thresholding! Binarization using Otsus method! Locally adaptive thresholding! Maximally stable extremal regions! Color-based segmentation! Region labeling and counting! Region moments!
Bernd Girod: EE368 Digital Image Processing!
Image Segmentation no. 1 !
Gray-level thresholding !
How can holes be lled?!
Original image! Peter f [x,y]!
Thresholded ! Peter m [x,y]

f [ x, y ] m [ x, y ]
Error probability for thresholding !
pdf!
Background!
Foreground!
error probability!
Gray level!
Optimal supervised thresholding !

MAP (Maximum A Posteriori) detector! pdf! Background!
Foreground!
error probability!
Gray level!
If different outcomes are associated with different costs: more general Bayes minimimum risk detector!
Unsupervised thresholding !
Idea: nd threshold T that minimizes within-class variance of both foreground and background (same as k-means)!
Equivalently, maximize between-class variance!
2 2 between T = 2 within T
()
()
x, yFgrnd
1 N Fgrd 1 2 2 = f x, y N N Fgrd N x, y = + = N Fgrnd

2
f x, y
2 Fgrnd
N Bgrnd 1 N N Bgrnd
x, yBgrnd
f x, y Bgrnd
2 2
N Fgrnd
()
+ = Fgrnd + N N N N T N Bgrnd T 2 Fgrnd T Bgrnd T N2
2 Fgrnd
N Bgrnd
2 Bgrnd
N Fgrnd
N Bgrnd
Bgrnd
()
()
( ))
[Otsu, 1979]!
Bernd Girod: EE368 Digital Image Processing! Image Segmentation no. 5 !
Unsupervised thresholding (cont.) !
Algorithm: Search for threshold T to maximize!
2 between
(T ) =
N Fgrnd T N Bgrnd T N
2
()
( )
Fgrnd
(T ) (T ))
Bgrnd
Useful recursion for sweeping T across histogram:!

N Fgrnd T + 1 = N Fgrnd T + nT N Bgrnd
Bgrnd T
Fgrnd Bgrnd
( ) () (T + 1) = N (T ) n (T ) N (T ) + n T (T + 1) = N (T + 1) (T ) N (T ) n T (T + 1) = N (T + 1)
Fgrnd Fgrnd T Fgrnd Bgrnd Bgrnd T Fgrnd
[Otsu, 1979]!
Sometimes, a global threshold does not work !
Original image!
Thresholded with Otsus Method!

Locally adaptive thresholding !

Slide a window over the image! For each window position, decide whether to do thresholding!

Can use variance or other suitable criterion! Thresholding should not be performed in uniform areas!
Non-uniform areas: apply Otsus method (based on local histogram) ! Uniform areas: classify the entire area as foreground or background based on mean value!
Locally adaptive thresholding (example) !
Original image!
Global threshold!
Local threshold!
Maximally stable extremal regions !
Extremal region: any connected region in an image with all pixel values above (or below) a threshold! Observations:!
Nested extremal regions result when the threshold is successively raised (or lowered).! The nested extremal regions form a component tree.!
Key idea: choose thresholds such as the resulting bright (or dark) extremal regions are nearly constant when these thresholds are perturbed by +/-

maximally stable extremal regions (MSER)"

[Matas, Chum, Urba, Pajdla, 2002]!
MSERs: illustration !
A+! A!
+! ! -!
A-!
Local minimum of
A A + MSER A
[Matas, Chum, Urba, Pajdla, 2002]!
Component tree of an image !

Image f [x, y]
level set
level set
level set
level set
f [x, y] > 0
f [x, y] > 1
f [x, y] > 2
f [x, y] > 3
[example from Najman, Couprie, 2006]! Bernd Girod: EE368 Digital Image Processing!
Areas of nested extremal regions !

A0 = 49
A1 = 11
A1 = 8
A2 = 5
A2 = 4
A2 = 4 A3 = 1
A3 = 2
A A + Local minima of sequence ; = , + 1, MSERs A

[example from Najman, Couprie, 2006]! Bernd Girod: EE368 Digital Image Processing! Image Segmentation no. 15 !
MSER: examples !
Dark MSERs, =15 !
Original image !
Bright MSERs, =15!
Chroma keying !

Color is more powerful for pixel-wise segmentation: 3-d vs. 1-d space! Take picture in front of a blue screen (or green, or orange)!
Landsat image processing !
Original Landsat image! false color picture out of bands 4,5,6!

Water area segmented and enhanced! to show sediments !

Source: US Geological Survey USGS, http://sfbay.wr.usgs.gov/
Multidimensional MAP detector !
Label categories in training set by hand! Subdivide n-dimensional space into small bins! Count frequency of occurrence for each bin and class in training set, label bin with most probable class! (Propagate class labels to empty bins)! For test data: identify bin, detect the more probable category!
MAP detector in RGB-space !
Original image!
Skin color detector!
[S. Leahy, EE368 class project, 2003]!
Linear discriminant function !
To segment image with n components fi, i=1,2,,n into two categories, perform test!
Categories are separated by hyperplane in n-space! Numerous techniques to determine weights see, e.g., [Duda, Hart, Stork, 2001] ! Can be extended to the intersection of several linear discrimant functions! Can be extended to multiple categories!
wi, i=0,1,2,,n
Region labeling and counting !
How many bacteria in this picture?!
Original Bacteria image!

after thresholding!
Which pixels belong to the same object (region labeling)?! How large is each object (region counting)?!
4-connected and 8-connected neighborhoods !
Denition: a region is a set of pixels, where each pixel can be reached from any other pixel in the region by a nite number of steps, where each step starts at a pixel and ends in the neighborhood of the pixel. !
4-neighborhood!
8-neighborhood!
Typically, leads to the same regions, except when a region is only connected across diagonally adjacent pixels.!
Region labeling algorithm (4-neighborhood) !

Loop through all pixels f [x,y], left to right, top to bottom! If f [x,y]=0, do nothing.! If f [x,y]=1, distinguish 4 cases!
Generate! new region! label!
Copy label! from above!
Copy label! from the left!
Copy label from the left. If labels above and to the left ! are different, store ! equivalence. !
Second pass through image to replace equivalent label by the same label.!
Example: region labeling !
Region counting algorithm !

Measures the size of each region! Initialize counter[label]=0 for all label
Loop through all pixels f [x,y], left to right, top to bottom! If f [x,y]=0, do nothing.! If f [x,y]=1, increment counter[label[x,y]]

Small region removal !
Loop through all pixels f [x,y], left to right, top to bottom! If f [x,y]=0, do nothing.! If f [x,y]=1 and counter[label[x,y]]<S, set f [x,y]=0! Removes all regions smaller than S pixels!
Hole lling as dual to small region removal !

Mask with holes! After NOT operation,! (background) region labeling,! small region removal,! and second NOT operation!
Region Moments !
Raw moments! Central moments!
M pq =
x,yRegion
x p yq
pq =
x,yRegion
(x x ) (y y )
M 10 M 01 with x = and y = M 00 M 00
Region orientation and eccentricity: calculate eigenvectors of covariance matrix! 20 11

11 02
Example: Detecting bar codes !

Original Image !

Locally adaptive binarization !

Locally adaptive binarization ! Filtering by eccentricity!

Locally adaptive binarization ! Filtering by eccentricity! Filtering by major axis length !

Locally adaptive binarization ! Filtering by eccentricity! Filtering by major axis length ! Filtering by orientation !

Segmentation

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Segmentation

Uploaded by

Copyright:

Available Formats

Chapter Overview: Image Segmentation !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 1 !

Original image! Peter f [x,y]!

Thresholded ! Peter m [x,y]

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 2 !

Error probability for thresholding !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 3 !

Optimal supervised thresholding !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 4 !

Equivalently, maximize between-class variance!

1 N Fgrd 1 2 2 = f x, y N N Fgrd N x, y = + = N Fgrnd

+ = Fgrnd + N N N N T N Bgrnd T 2 Fgrnd T Bgrnd T N2

Unsupervised thresholding (cont.) !

Algorithm: Search for threshold T to maximize!

Useful recursion for sweeping T across histogram:!

Unsupervised thresholding (cont.) !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 7 !

Unsupervised thresholding (cont.) !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 8 !

Sometimes, a global threshold does not work !

Thresholded with Otsus Method!

Locally adaptive thresholding !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 10 !

Locally adaptive thresholding (example) !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 11 !

Maximally stable extremal regions !

maximally stable extremal regions (MSER)"

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 12 !

[Matas, Chum, Urba, Pajdla, 2002]!

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 13 !

Component tree of an image !

Areas of nested extremal regions !

A A + Local minima of sequence ; = , + 1, MSERs A

Dark MSERs, =15 !

Bright MSERs, =15!

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 16 !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 17 !

Landsat image processing !

Original Landsat image! false color picture out of bands 4,5,6!

Water area segmented and enhanced! to show sediments !

Image Segmentation no. 18 !

Multidimensional MAP detector !

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 19 !

MAP detector in RGB-space !

Skin color detector!

[S. Leahy, EE368 class project, 2003]!

Bernd Girod: EE368 Digital Image Processing!

Image Segmentation no. 20 !

Linear discriminant function !

Region labeling and counting !

How many bacteria in this picture?!