Superpixels Using Morphology for Rock Image

Segmentation
Sree Ramya s.

P.

Malladi, Sundaresh Ram and Jeffrey J. Rodriguez

Department of Electrical and Computer Engineering, University of Arizona, Tucson, AZ, USA
Email: {rmalladi.ram.jjrodrig}@email.arizona.edu

Abstract-Detection and segmentation of rocks is an important

first task in many

applications such as geological analysis,

and mining processes. Rocks are usually

planetary science
segmented using a variety of features such as texture, shading,
shape and edges. It is easier to compute these features for rock
superpixels rather than every pixel in the image. A superpixel
is a group of spatially coherent pixels that form a meaningful
homogeneous region, usually belonging to the same object. In this
paper, we perform a comparative study of some of the current
superpixel algorithms on rock images with regard to their ability
to adhere to image boundaries, their speed, and their impact
on rock segmentation performance. Also, we propose a new and
very simple superpixel algorithm, Superpixels Using Morphology
(SUM), which permutes a watershed transformation approach to
efficiently generate superpixels. We show that SUM achieves a
performance comparable to the recent superpixel algorithms on
the rock images.
Index Terms-morphology, watershed segmentation, area clos
ing, superpixels, rock particles.

I. INTRODUCTION
Detection and segmentation of rock particles IS Important
in order to measure the size distribution of rock particles
in mining processes to monitor the blasting quality, optimize
the blast design, and reduce costs and environmental impact.
Also, rock shape, weathering, and dispersion carry important
information about environmental characteristics and need to
be identified for efficient route planning in planetary science.
Superpixel segmentation is an attempt to capture the low-level
details in image by grouping the pixels such that they pro
vide spatial support for extracting features. Superpixels show
benefit in applications such as object tracking [1], detection
[2], segmentation [3], depth estimation [4] and object-based
compression.
Rock image segmentation can be thought of as a two
step process. First, superpixels are computed, and then a
region merging scheme is used to merge the superpixels into
a final segmentation. In this paper, we focus on computing
superpixels. The superpixels computed for the rock images
are expected to have the following properties:
1) Superpixel boundaries should accurately represent, the
edges of the rocks.
2) A superpixel should not include portions of more than
one rock. (Because post-processing is expected to in
clude a merging step, it is easier to recover from
oversegmentation than undersegmentation.)
3) The technique for computing superpixels should be
computationally simple and memory efficient.

978-1-4799-4053-0114/$31.00 2014 IEEE

Many algorithms have been recently developed to divide an

image into superpixels [5]-[8], [10]-[15]. We compare four
algorithms: normalized cuts (Ncuts) [5], turbopixels [12],
simple iterative linear clustering (SLIC) [13], entropy rate
superpixels (ERS) [14] and the proposed method, superpixels
using morphology (SUM). Ncuts treats the image as a graph
G
(V, E), where the vertices V represent the pixel locations
and the edges E represent the relation between the pixels (ver
tices). This graph is partitioned using contour and texture cues,
globally minimizing a cost function defined on the edges at
the partition boundaries. Run time for Ncuts is relatively large.
Turbopixels uses a level-set based geometric flow to dilate a set
of initially placed seeds defined by the user. This algorithm re
lies on other algorithms of varying complexity and sometimes
exhibits relatively poor adherence to the boundaries. SLIC is
a variation of k-means clustering for generating superpixels.
SLIC optimizes the distance calculations by limiting the search
space, and uses a weight parameter to control the compactness
of superpixels. For SLIC the segmentation accuracy critically
depends on how well this weight parameter is tuned. ERS
is a graph topology selection method where pixels and their
pairwise relations are respectively mapped to the vertices and
edges in the graph. Superpixels are then formed via graph
topology by maximizing an objective function. The objective
function has two components: (1) the entropy rate which
favors compact and homogeneous superpixels and (2) the
balancing term which favors superpixels with similar sizes.
ERS produces a segmentation with reasonable accuracy, but
the run-time may be too long for many applications.
=

In this paper, we describe a new superpixel segmentation

method using a modified watershed segmentation algorithm.
The algorithm is composed of three key steps: (1) compute
the magnitude gradient of the original image, (2) perform an
area closing operation on this magnitude gradient image, and
(3) apply a watershed transformation to the resultant image to
obtain the desired superpixels.

II. SUPERPIXELS USING MORPHOLOGY

We describe a new method for computing the superpixels in
an image, using morphology-based operators, which is faster
than the existing algorithms and is very memory efficient. The
new method, superpixels using morphology, is adapted from
the watershed segmentation algorithm [9].

145

SSIAI2014

Fig. I. (a) Original image. (b) Morphological gradient image (inverted) with a 3 x 3 square. (c) Closing of gradient magnitude image (inverted) with a disk
(r = 10). (d) Area closing of gradient magnitude image (inverted) (0:
100). (e) Watershed lines obtained from image (c) superimposed onto original image.
(f) Watershed lines obtained from image (d) superimposed onto original image.
=

A. Image Gradient

where,

Object boundaries are often characterized by intensity tran

sitions (edges) in the image. Various gradient operators are
widely used in image processing to detect these edges, the
basic principle being that large gradients indicate points where
there is a rapid intensity change. We compute the morpholog
ical gradient of the input image f defined by

gradb(f)

(f

EB

b) - (f e b),

(1)

where b is a structuring element, usually symmetric and having

a short support, EB is a morphological dilation operation, and
e is a morphological erosion operation. We use a 3 x 3
square-shaped structuring element b for computing the mor
phological gradient image. Fig. lea) shows an example image,
and Fig. l(b) shows the corresponding morphological gradient
magnitude image.
B.

Morphological Area Closing

The morphological gradient image may consist of spurious

strong local gradients within a single rock region due to gray
level variations, in addition to having true strong gradients near
the rock edges. A classical approach to suppress such spurious
gradients in the gradient image is the use of the morphological
closing operator. When there is no prior infonnation about
the shape of an object in an image, morphological closing
is usually perfonned with a disk-shaped structuring element
to preserve isotropy. Fig. l(c) shows the effect of applying
morphological closing to the gradient image in Fig. l(b). The
closing operation is then followed by watershed segmentation
to obtain the segmented image. However, artifacts may appear
in the segmented image. Fig. lee) shows an example of such
artifacts, where the crest lines have moved many pixels away
from the actual boundary of the object. The extent of the
deviation depends on the filtering strength (i.e., the radius of
the disk).
In order to suppress the spurious gradients in the morpho
logical gradient image, along with the associated segmentation
artifacts, we employ Vincent's morphological area closing
operator [17]. The gray-scale formulation of this operator
relies on the threshold superposition principle and is given
by
(2)

BBe,a =

{X

E: X is Be-connected, Area(X) 2 o:}

This operator removes connected components whose area

is smaller than a given area parameter 0:. This morphological
filter is shape preserving because it acts on connected compo
nents and, therefore, does not typically change the shape of
the structures in the image. We use a fast implementation of
this operator by Meijster and Wilkinson [18]. Fig. led) shows
the result of applying the area closing operator to the gradient
image in Fig. l(b).

C. Watershed Segmentation
The watershed transformation [9] is a popular segmentation
algorithm, which divides the gray-level image into regions
that are each associated with one local minimum. Consider
the gray-level image as a topographic map. For each regional
minimum of this map, define a catchment basin (i.e., a
region) as all those points whose steepest-slope paths reach
this minimum. The watershed lines are then defined as the
closed one-pixel-thick crest lines that separate the adjacent
catchment basins. Due to numerous local minima present
within an image, applying watershed segmentation directly
to the image ends up in extreme over-segmentation. We
apply the watershed segmentation algorithm to the area-closed
gradient image to obtain the desired superpixels. Fig. 1(0
shows the watershed lines obtained by applying the watershed
segmentation algorithm to the area-closed gradient image in
Fig. led), superimposed onto the original image.
III.

PERFORMANCE EVALUATION

To analyze the quality of superpixel segmentation, two met

rics are used: under-segmentation error and boundary recall.
Manually segmented "ground truth" images are used as a
reference to compute the metrics.

A. Under-Segmentation Error (U)

Under-segmentation error [16] measures false merging of
superpixels across the ground truth borders. A superpixel
is considered to be falsely merged if it spans across a
ground truth border. Consider a ground truth segmentation
comprising M segments {gl' g2'
, 9 M} and a corresponding
automatic superpixel segmentation comprising L segments
{8l' 82,
8 L}' Let N be the number of pixels in the image,
.

146

TABLE

RUN TIME IN SECONDS FOR DIFFERENT ALGORITHMS

# of
Segments
-TP
-sue

.. . . . . . . . . . . . . . . . . . . . . .

-ERS
" 'SUM
-Nellts
500

1()()()
1500
2()()()
Number of Segments

2500

Fig. 2. (a). Under-segmentation error (b). Boundary recall.

and let the operator 11 represent the size of a segment in pixels.

The under-segmentation error is a value within the range [0,1],
with 0 meaning no under-segmentation error, and is defined
as
M

u=

L
i=l

min{lsjl-lsjngil,lsjngil}-M

{SjICISjngil)",}

(3)

B. Boundary Recall ()

Boundary recall measures the fraction of ground truth

boundaries that fall within a fixed distance from a superpixel
boundary. Consider a ground truth segmentation GT and a
superpixel segmentation S. Let TP represent the number of
boundary pixels in GT that have a boundary pixel in S within a
distance of 2 pixels. Let FN represent the number of boundary
pixels in GT for which there does not exist a boundary pixel
in S within a distance of 2 pixels. Boundary recall is a value
in the range [0,1]' and is defined as
=

C.

TP
TP+FN

(4)

Experiments

The recent superpixel algorithms [5], [12], [13], [14] and the
proposed method SUM were tested on a set of 10 rock images.
Each image has size 480 x 640 pixels. A careful manual
segmentation of the rock images was considered as ground
truth for all subsequent analysis. On average, each rock image
has around 50 to 100 ground truth rock regions. The dataset of
rock images includes rocks with varying illumination, shading,
shape, and texture. The goal of the superpixel algorithms
should be to produce a minimum number of superpixels with
good segmentation quality (low under-segmentation error and
high boundary recall). The run time of the algorithms is also
an important factor.
We used open source implementations of the superpixel
algorithms available online. The original implementation of
the Ncuts algorithm resizes the image to 160 x 160 for faster
compution. We disable the image resizing poperty of Ncuts
algorithm and keep the size of the image fixed for all the
methods to have a fair comparison. The number of superpixels
is the only parameter used by the turbopixel algorithm. SLIC
has two parameters: region size (to produce uniformly sized

Turbo

ERS

SLIC

Ncuts

SUM

25

14.3 10

2.045

0.650

158.349

0.020

500

39. 1 10

2.9 10

1.699

1802.600

0.024

1000

4 1.20 1

3.64 1

1.73 1

0.027

2500

44.036

5.74 1

1.960

0.044

regions) and regularizer (to control the compactness). The

region size was fixed, and the regularizer that gave the least
under-segmentation error was chosen. The region sizes were
chosen so as to give the same number of segments as the
other methods for fair comparison. ERS has four parameters:
number of superpixels, weighting factor for the balancing
function, Gaussian kernel parameter, and connectedness (4connected or 8-connected). The number of superpixels was
fixed, and the combination of other parameters that gave
the least under-segmentation error was chosen. For the rock
images tested, the optimal range of superpixels lies between
250 and 500. Thus, we compared the performance of all the
algorithms in this operating range.
The under-segmentation error measures the amount of false
merging, so we want to minimize this error. Fig. 2(a) shows
a plot of the under-segmentation error vs. the number of
superpixels for all the automated methods. In the range of 250500 superpixels, the difference between the SUM algorithm
and the other methods is very small. The under-segmentation
error of Ncuts and ERS is 0.03 less than SUM. The under
segmentation error of SLIC and the turbopixel algorithm is
just 0.007 less than SUM.
Boundary recall measures the adherence of superpixel
boundaries to ground truth image boundaries, so we want to
maximize this quantity. Fig. 2(b) shows that the boundary re
call of SUM is comparable to the other superpixel algorithms.
The boundary recall of SUM is 0.2 greater than turbopixels.
The boundary recall of ERS, SLIC, and Ncuts is 0.07 greater
than SUM.
All the automated algorithms were run on a 2.5 GHz Intel
core is processor with 4 GB RAM. Table I compares the run
time of all the algorithms. SUM outperforms all the algorithms
under study with respect to run time. In the operating range,
SUM is 1.674 seconds faster than SLIC and 2.886 seconds
faster than ERS. The algorithms are ranked in the following
order with respect to run time: SUM, SLIC, ERS, Turbopixels,
and Ncuts.
Fig. 3 shows the superpixel segmentation results for an
example rock image. Typically, the superpixel segmentation
would next undergo post-processing in order to merge the
oversegmented regions.
IV.

CONCLUSION

In this paper, we compared recent superpixel segmentation

algorithms on rock images. The Ncuts algorithm gives compa
rable results to the rest of the automated algorithms in tenns

147

(d)

(e)

(f)

Fig. 3. (a) Original image. Results of the automated algorithms: (b) Ncuts, (c) turbopixels, (d) SLIC, (e) ERS, (f) SUM (proposed method).

of under-segmentation and boundary recall, but requires more

computation time. ERS and SUC perform well in terms of
under-segmentation and boundary recall and are also faster
than turbopixels and Ncuts. SUM is the fastest among all the
algorithms and simple to implement when compared to the
other methods. At the same time, its under-segmentation and
boundary recall are comparable to ERS and SUe. Next, we
plan to implement various region merging schemes to combine
with these superpixe\ algorithms in order to determine which
algorithm gives the most accurate final segmentation.
ACKNOWLEDGMENT

We would like to thank Split Engineering LLC for providing

the rock image data set.
REFERENCES

[I] S. Wang, H. Lu, L. Yang, and M-H. Yang, "Superpixel tracking", in Proc.
IEEE Int. Con! Computer Vision, pp. 1323-1330,201l.
[2] B. Fulkerson, A. Vedaldi, and S. Soatto, "Class segmentation and object
localization with superpixel neighborhoods." in Proc. IEEE Int. COIll
Computer Vision, pp. 670-677,2009.
[3] p. Mehrani and O. Veksler, "Saliency segmentation based on leaming
and graph cut refinement." in Proc. British Machine Vision COIif., pp.
110.1-110.12,2010.
[4] B. Micusfk and J. Kosecka, "Multi-view superpixel stereo in urban
environments," Int. J. Computer Vision vol. 89,no. I,pp. 106-119,Aug.
2010.
[5] J. Shi and J. Malik, "Normalized Cuts and [mage Segmentation," IEEE
Trans. Pattern. Anal. and Mach. buell., vol. 22,no. 8,pp. 888-905,Aug.
2000.
[6] P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient graph-based image
segmentation;' Int'l. J. Computer Vision vol. 59,no. 2,pp. 167-181,Sep.
2004.
[7] D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature
space analysis;' IEEE Trans. Pattern. Anal. Mach. buell., vol. 24, no. 5,
pp. 603-619,May 2002.

148

[8] A. Vedaldi and S. Soatto, "Quick shift and kernel methods for mode
seeking;' in Proc. European COIif. Computer Vision, pp. 705-718,2008.
[9] L. Vincent and P. SoilIe, "Watersheds in digital spaces: an efficient
algorithm based on immersion simulations," IEEE Trans. Pattern. Anal.
Mach. Intell., vol. 13, no. 6, pp. 593-598,Jun. 1991.
[10] O. Veksler, Y. Boykov, and P. Mehrani, "Superpixels and supervoxels in
an energy optimization framework," in Proc. European COlif. Computer
Vision, pp. 211-224,2010.
[II] M. Van den Bergh, X. Boix, G. Roig, B. de Capitani, and L. Van
Gool, "SEEDS: superpixels extracted via energy-driven sampling, in Proc.
European COIll Computer Vision, pp. 213-26,2012.
[12] A. Levinshtein, A. Stere, K. N. Kutulakos, D. J. Dickinson, and K.
Siddiqi, "Turbopixels: Fast superpixels using geometric flows," IEEE
Trans. Pattern. Anal. Mach. buell., vol. 31, no. 12, pp. 2290-2297,Dec.
2009.
[13] R. Achanta, A. Shaji, A. Lucchi, P. Fua, and S. Susstrunk, "SLIC
superpixels compared to state-of-the-art superpixel methods," IEEE Trans.
Pattern. Anal. Mach. buell., vol. 34, no. 11, pp. 2274-2282,Nov. 2012.
[14] M. Y. Liu, O. Tuzel, S. Ramalingam, and R. Chellappa, "Entropy-rate

clustering: cluster analysis via maximizing a submodular function subject

to a matroid constraint," IEEE Trans. Pattern. Anal. Mach. buell., vol.
36, no. 1, pp. 99-112,Jan. 2014.
[15] C. Conrad, M. Mertz, and R. Mester, "Contour-related superpixels;' in

Proc. Int. COIll on Energy Minimzation Methods in Computer Vision and

Pattern Recognition, pp. 280-293,2013.
[16] P. Neubert and P. P rotzel, "Evaluating superpixels in video: metrics
beyond figure-ground segmentation," in Proc. British Machine Vision
Conference, pp. 1-11,2013.
[17] L. Vincent, "Grayscale area openings and closings: their applications
and efficient implementation," in EURASIP Workshop on Mathematical
Morphology and its Applications to Signal Processing, pp. 22-27,1993.
[18] A. Meijster and M. H. F. Wilkinson, "A comparison of algorithms for
connected set openings and closings;' IEEE Trans. Pattern. Anal. Mach.
buell., vol. 24, no. 4, pp. 484-494,Apr. 2002.