CMM-ITWM Joint Meeting, Fontainebleau

1
Serge BEUCHER Serge BEUCHER
Centre de Centre de Morphologie Morphologie Math Math matique matique
Mines Mines Paristech Paristech
Morphological Image Segmentation:
Morphological Image Segmentation:
A User
A User

s Guide
s Guide
CMM-ITWM Joint Meeting, Fontainebleau
2
It It s a flooding process s a flooding process
Flooding sources are the Flooding sources are the
minima of the function minima of the function
The result is a partition of the The result is a partition of the
image into catchment basins and image into catchment basins and
watershed lines (dams). watershed lines (dams).
Watershed
Watershed
Transformation
Transformation
Efficient Efficient algorithms algorithms exist exist to to
realise realise rapidly rapidly this this operator operator. .
CMM-ITWM Joint Meeting, Fontainebleau
3
Use of the watershed transform for Use of the watershed transform for greyscale greyscale image segmentation: image segmentation:
the initial idea the initial idea
The watershed lines of the
gradient corresponds to
the contour lines
Using
Using
the
the
Watershed
Watershed
Transformation
Transformation
Catchment basins correspond to
homogeneous grey regions in the
image.
CMM-ITWM Joint Meeting, Fontainebleau
4
The gradient watershed is over The gradient watershed is over- -segmented. segmented.
To avoid this over To avoid this over- -segmentation due to segmentation due to
numerous sources of flooding, one can numerous sources of flooding, one can
select some of them (the markers) and select some of them (the markers) and
perform perform a marker a marker- -controlled watershed controlled watershed
transform transform. .
Gradient images are noisy
and contain many minima.
Each minimum generates a
catchment basin in the WTS.
Coping
Coping
with
with
Over
Over
-
-
Segmentation
Segmentation
CMM-ITWM Joint Meeting, Fontainebleau
5
original image original image gradient gradient
Markers corresponding to Markers corresponding to
the two regions to be the two regions to be
segmented (drawn by segmented (drawn by
hand actually) hand actually)
Advantages of the
Advantages of the
Marker
Marker
-
-
Controlled Watershed
Controlled Watershed
gradient watersheds gradient watersheds
CMM-ITWM Joint Meeting, Fontainebleau
6
A
A
Simple
Simple
User
User

s
s
Guide
Guide
A morphological segmentation A morphological segmentation
process is performed in two process is performed in two
steps steps
The function f quantifies the The function f quantifies the
criterion which is used by the criterion which is used by the
segmentation segmentation
The markers indicate the The markers indicate the
regions/objects to be extracted regions/objects to be extracted
This scheme (segmentation paradigm) has the advantage to be This scheme (segmentation paradigm) has the advantage to be
generic and to be applicable to many segmentation problems (2D, generic and to be applicable to many segmentation problems (2D,
3D, 3D, greytone greytone, , color color, multi , multi- -spectral, interactive segmentation, etc.). spectral, interactive segmentation, etc.).
It It s, however, a simplistic user s, however, a simplistic user s guide s guide
CMM-ITWM Joint Meeting, Fontainebleau
7
Which
Which
Criteria?
Criteria?
Which
Which
Markers?
Markers?
Regarding Regarding greyscale greyscale (or (or color color) images, ) images, contrast contrast criteria criteria are are used used
and and therefore therefore functions functions quantifying quantifying differences differences between between adjacent adjacent
pixels and/or pixels and/or regions regions: :
Gradient Gradient
Top Top- -hat hat transform transform
Various Various combinations combinations
The The watershed watershed approach approach can can also also be be used used to segment sets to segment sets
according according to to their their shapes shapes and and sizes sizes. In . In this this case, the case, the Distance Distance
function function is is widely widely used used. .
Markers are Markers are built built by by various various means means. . They They are are often often obtained obtained from from
extrema (minima or maxima) of the extrema (minima or maxima) of the criteria criteria functions functions or by more or by more
sophifisticated sophifisticated approaches approaches using using a a wide wide range of range of morphological morphological tools tools
(filters, geodesic operators). (filters, geodesic operators).
CMM-ITWM Joint Meeting, Fontainebleau
8
Coffee grains Coffee grains
The distance function of the set is computed.
This distance function is inverted and its
watershed is performed. The marker set is
made of the maxima of the distance function.
Set
Set
Segmentation,
Segmentation,
an
an
Example
Example
CMM-ITWM Joint Meeting, Fontainebleau
9
Silver nitrate grains on a film Silver nitrate grains on a film
Short
Short
Anthology
Anthology
of Applications
of Applications

3D restitution of water drops from an hologram 3D restitution of water drops from an hologram
CMM-ITWM Joint Meeting, Fontainebleau
10
3D brain NMR image 3D brain NMR image

With
With
3D
3D
Images
Images
Polyester foam Polyester foam
CMM-ITWM Joint Meeting, Fontainebleau
11
Lane detection based on a Lane detection based on a
watershed segmentation applied watershed segmentation applied
on each image of the sequence on each image of the sequence

And
And
With
With
Animated
Animated
Images
Images
The PROMETHEUS project: The PROMETHEUS project: road segmentation and obstacle detection road segmentation and obstacle detection
Lanes detection with re Lanes detection with re- -use of use of
the previous result as markers the previous result as markers
in the current image in the current image
CMM-ITWM Joint Meeting, Fontainebleau
12
Example of traffic lanes segmentation: the lanes are not separat Example of traffic lanes segmentation: the lanes are not separated by ed by
a significant difference in grey levels. Therefore, using contra a significant difference in grey levels. Therefore, using contrast st
criteria is irrelevant. Conversely, to use shape or size criteri criteria is irrelevant. Conversely, to use shape or size criteria, we need a, we need
to work on the set corresponding to the road, which must be obta to work on the set corresponding to the road, which must be obtained ined
by a by a segmentation. segmentation.
Extending
Extending
Shape
Shape
Criteria
Criteria
to
to
Greyscale
Greyscale
Images,
Images,
Is
Is
It
It
Possible?
Possible?
Until recently, it was difficult to apply shape and size criteri Until recently, it was difficult to apply shape and size criteria to a to
greyscale image segmentation. greyscale image segmentation.
New operators allow to bridge the gap between these two kinds of New operators allow to bridge the gap between these two kinds of
criteria. criteria.
CMM-ITWM Joint Meeting, Fontainebleau
13
Numerical
Numerical
Residues,
Residues,
Short
Short
Introduction
Introduction
Starting Starting from from two two sequences sequences of transformations of transformations
( )
i i
I i
Sup =

( ) 1 max arg + =
i i
q
i i

i
and and with with
, , we we define define a doublet of a doublet of operators operators: :
The The residual residual transformation transformation
Its Its associated associated function function
i

An elementary residual operator is defined by the difference of two

operators.
Many residual operators exist. Many of them Many residual operators exist. Many of them
are used as criteria functions in segmentation: are used as criteria functions in segmentation:
morhological morhological gradient, top gradient, top- -hat for numerical hat for numerical
images, ultimate erosion and distance for sets images, ultimate erosion and distance for sets
General definition General definition
CMM-ITWM Joint Meeting, Fontainebleau
14
New
New
Residues
Residues
1 +
=
=
i i
i i


1 +
=
=
i i
i i


q q is is called called Quasi Quasi- - Distance Distance.
is is named named Ultimate Ultimate Opening Opening
q q is is the the Granulometric Granulometric function function
}
}
Thanks to this general definition of a residual transform, it is Thanks to this general definition of a residual transform, it is possible possible
to extend to functions residues defined for sets: to extend to functions residues defined for sets:
It is also possible to define new transformations: It is also possible to define new transformations:
( )
i rec i
i i
=
=
Ultimate Erosion Ultimate Erosion
CMM-ITWM Joint Meeting, Fontainebleau
15
Heap Heap of rocks of rocks Ultimate Ultimate Opening Opening
Granulometric Granulometric function function
Ultimate
Ultimate
Opening
Opening
Granulometic
Granulometic
Function
Function
CMM-ITWM Joint Meeting, Fontainebleau
16
Each Each threshold threshold of the of the
granulometric granulometric function function q q is is
eroded eroded by a by a disk disk of size k of size k (k<1) (k<1)
This This operation operation produces produces markers of markers of
blocks blocks whose whose size size is is proportional proportional
to the size of the block. As a to the size of the block. As a result result, ,
markers are markers are better better centered centered, , even even
small small particles particles are are well well marked marked and and
the the watershed watershed segmentation segmentation is is of of
better better quality quality. .
Markers
Markers
Generation
Generation
The The granulometric granulometric function can be function can be
used to produce efficient markers used to produce efficient markers
for the watershed segmentation for the watershed segmentation
CMM-ITWM Joint Meeting, Fontainebleau
17
Some
Some
Results
Results
of
of
Segmentation
Segmentation
CMM-ITWM Joint Meeting, Fontainebleau
18
Ultimate
Ultimate
Opening
Opening
Can
Can
Be
Be
Defined
Defined
From
From
Any
Any
Opening
Opening
Operator
Operator
For instance, with opening by geodesic For instance, with opening by geodesic reconstruction reconstruction
This operator emphasizes the size criterion . This operator emphasizes the size criterion .
Residual operators provide efficient non parametric filters. The Residual operators provide efficient non parametric filters. The Ultimate Ultimate
Opening is a remarkable tool Opening is a remarkable tool for marking and for marking and extracting extracting salient salient features features
from an image. from an image.
Initial image Initial image
Ultimate Opening Ultimate Opening
by reconstruction by reconstruction
Granulometric Granulometric
function function
CMM-ITWM Joint Meeting, Fontainebleau
19
Quasi
Quasi
-
-
Distance and Segmentation
Distance and Segmentation
A quasi A quasi- -distance distance computed computed on a on a greyscale greyscale image image provides provides the the sizes sizes of of
the flat ( the flat (homogeneous homogeneous) ) regions regions Markers for a segmentation Markers for a segmentation based based
on size and on size and geometry (convexity). geometry (convexity).
Quasi Quasi- -distances distances performed performed both both on the on the
image and the image and the complementary complementary one one
d, d d, d
Sup of the Sup of the results results h=sup(d,d h=sup(d,d ) )
Markers extraction (maxima or Markers extraction (maxima or threshold threshold) )
Watershed Watershed of h of h
CMM-ITWM Joint Meeting, Fontainebleau
20
Segmentation
Segmentation
With
With
Quasi
Quasi
-
-
Distances
Distances


WS
CMM-ITWM Joint Meeting, Fontainebleau
21
Video Video surveillance surveillance
scene scene
Another
Another
Example
Example
Quasi Quasi- -Distances Distances
Markers Markers Segmentation Segmentation
CMM-ITWM Joint Meeting, Fontainebleau
22
Gradient
Gradient
and
and
Quasi
Quasi
-
-
Distance
Distance
Quasi Quasi- -distance distance can can be be computed computed on the on the inverted inverted gradient gradient function function
Only Only one quasi one quasi- -distance distance is is calculated calculated
Hierarchy Hierarchy of of regions regions based based on on their their relative relative contrast contrast
The The shape shape of of regions regions is is taken taken into into account account ( (closure closure of of imperfectly imperfectly
closed closed regions regions) )
CMM-ITWM Joint Meeting, Fontainebleau
23
In the initial user In the initial user s guide, no clue was given about the morphological s guide, no clue was given about the morphological
operators which could be used for segmentation. operators which could be used for segmentation.
We have shown that criteria functions belong to the residual tra We have shown that criteria functions belong to the residual transforms nsforms
class and that markers are linked to extrema of these functions. class and that markers are linked to extrema of these functions.
Residual transformations are built in such a way that they, not Residual transformations are built in such a way that they, not only, only,
emphasize variations of some features (contrast, size, shape, et emphasize variations of some features (contrast, size, shape, etc.) but c.) but
also, they indicate where the greatest variations occur, what ar also, they indicate where the greatest variations occur, what are their e their
amplitudes and which index value produces them. amplitudes and which index value produces them.
New residual transforms have been introduced. They bridge the ga New residual transforms have been introduced. They bridge the gap p
between the numerical and binary images regarding shape and size between the numerical and binary images regarding shape and size
criteria. criteria.
Other residues (which have not been presented here) are also ver Other residues (which have not been presented here) are also very y
efficient: regularised gradients,spatial efficient: regularised gradients,spatial- -temporal gradients, critical balls, temporal gradients, critical balls,
pilings (in hierarchical segmentations), etc. pilings (in hierarchical segmentations), etc.
Conclusions and Perspectives
Conclusions and Perspectives