2015 4th International Conference on Advanced Computer Science Applications and Technologies
An Efficient Object Extraction with Graph-Based Image Segmentation
Ali Saglam
Department of Computer Engineering
Faculty of Engineering
Seluk University, Konya, Turkey
alisaglam@selcuk.edu.tr
Abstract Object extraction process is a closely related issue
with image segmentation process. To separate an image to
several segments formed similar pixels, many methods are
proposed in the area of image processing. Graph-based image
segmentation is also one of the segmentation methods. Because
of their representation convenience and ease of use, graphs are
used as important tools in many image processing applications.
While an image segmentation process runs, the processes
splitting a graph to sub graphs and merging sub graphs are
carried out in the meanwhile. To fulfill these processes, the
method uses some local features such as differences between
vertices in the graph, which represent pixels, or global features
of the image and its segments. To extract an object from an
image, we first segmented the entire image, because of to save
global features, or to obtain more accurate segmentation.
Finally, we extract the intended object from the image by
merging the segments that are inside the area drawn before by
us. We test the method on some images in the Segmentation
Evaluating Database from Weizmann Institute of Science and
evaluate the segmentation results. Our F-measure score values
show that it seems noticeable good segmentation.
Keywords- Segmentation; Object Extraction; Graph;
Minimum Spanning Tree
I. INTRODUCTION
In many image processing application such as object
extraction, medical image processing, satellite image
processing, video processing, object recognition, image
reconstruction and more, image segmentation is an important
stage [1,2,3]. In the segmentation stage, an image is
partitioned into regions according to similarity of its pixels
[4,5,6]. Image segmentation makes an image and its
perception easier and more meaningful to analyze [7]. It
typically focuses on the object in the image and borders.
Image segmentation techniques try to cluster homogeneous
pixels within the same regions by using color, pattern
(texture) and density differences [8].
Many pattern recognition and computer vision
applications reduce image information for more efficient
segmentation by taking advantage of global features of
images such as eigenvectors, standard deviation, energy
function and so on [9]. However, in this way, the processing
times exceed the desired time period for many computer
visions application when the process is applied to especially
high-resolution images. Due to the computational complexity
of spectral clustering methods, intuitive approaches such as
greedy algorithms, which only make a decision according to
978-1-5090-0424-9/16 $31.00 2016 IEEE
DOI 10.1109/ACSAT.2015.29
Nurdan Akhan Baykan
Department of Computer Engineering
Faculty of Engineering
Seluk University, Konya, Turkey
nurdan@selcuk.edu.tr
the current condition, or MST based algorithms are preferred
more in real-time applications [8].
Many segmentation algorithms have been developed over
the years. Among the existing image segmentation methods,
many successful ones make use of mapping the image
elements onto a graph so that they solve the segmentation
problem in a spatially discrete space by the from graph
theory tools [10]. These methods are called graph-based
image segmentation methods.
In a graph, each vertex (node) indicates a point, and each
edge joins the points by a line. In a weighted graph, each
edge is associated with a real number called weight [11].
Minimum Spanning Tree (MST) based methods are also
one of the graph-based methods. To reach all vertices in a
graph, the structure of spanning tree is generally used,
because a spanning tree does not contain any cycle. If we
navigate through the nodes in a spanning tree without return
to back and we meet again the node that we met with before,
there is a cycle. In a graph, MST is the shortest spanning tree
that has the smallest total weight among spanning trees that
cover all vertices in the graph.
In 1960s, pioneering methods of graph theory emphasize
the basis of gestalt principles in perception meaningful
clusters according to similarities and differences between
vertices [10]. In this way, the graph is divided into sections
according to the local features. In fact, every part of the
image is considered to be an object. At this stage of the
graph-based approach, the process of image segmentation
has been an implicit transaction based on clustering and
MST [1]. These methods generally have been used for local
computations on the segmentation results. They do not
guarantee a well segmentation over all images, because of
taking into account only local features. To overcome this
problem, then, some methods that are recursive and also take
into account the characteristics of sub clusters are integrated
to MST-based methods [7,10,12,13,14,15].
In this paper, we use Boruvkas MST algorithm for MST
based clustering and use Felzenszwalb and Huttenlochers
equation (Eq. 3) as comparison criterion to merge vertices.
Then, we apply our graph-based object extraction method
upon this.
II. RELEATED WORKS
MST-based segmentation methods are associated with a
graph-based clustering basis. Common works on graph
clustering can be dated back to old times.
86
 In the beginning of 1970s, Zahn [16] applied a method
that removes the longest edges from the MST formed
to obtain clusters for predefined constant . At the same
time, to overcome very small cluster pieces, he proposes a
deletion criterion that compares the standard deviations on
either sides of the edge which will be deleted and weight of
the edge [17,18]. Lopresti and Zhou [19] used the Zahns
method on color image segmentation using Euclidean
distance measurement in 2000.
In 1980s, Morris et al. [20] used MST to partition
images hierarchically. The developed method carries a
segmentation process, which is in different scales, based on a
principle which requires the most similar pixels grouped
together and dissimilar pixels separated. It breaks the highest
weight of the MST, and the graph pieces are formed by
maximum differences between the neighboring sub graphs.
He applied a recursive MST algorithm as another method.
According to this algorithm, after each merging, the weights
of the differences of the neighboring clusters are updated and
the edge list is sorted every iteration. This form is thought to
be inefficient in terms of an algorithm speed. Kwok et al.
[14,15], then, proposed a fast recursive MST algorithm.
According to this algorithm, to speed up the process weights
of the edges are transformed a discrete structure, and instead
of the stack structure, heap structure is used. In subsequent
years, Bayramoglu applied this method on distributed
systems and showed comparison results [7].
Back to 1998, Felzenszwalb and Huttenlocher proposed
an efficient graph-based image segmentation method based
on the structure of Kruskals MST algorithm [12,21]. It uses
both global differences between neighboring two sub-graphs
and local differences inside each sub-graph. Moreover, it
nearly runs in linear time with the number of nodes in the
graph, and this provides an advantage for real-time video
segmentation which requires fast processing. Another MST-
based algorithm is developed by Haxhimusa and Kropatsch,
which makes up use the Boruvkas MST principles [22]. As
E refers the number of edges and N refers the number of
nodes which represent the pixels in image (Fig. 1),
computational complexity of Kruskal's algorithm is
and the complexity of Boruvkas algorithm is
[8].
Figure 1. Graph representation of pixels according to 8-neighborhood
87
In this study, the calculation formula of the number of
edge in an image is Eq. 1 for 8-connected neighborhood, as
and that refer to lengths of the dimension of the image.
(1)
In 2009, Wassenberg et al. [23] developed an algorithm
that compute the independent MSTs and merge them. In
2010, Grundmann et al. [4] proposed a Graph-based Video
Segmentation technique based on Graph-based Image
Segmentation of Felzenszwalb et al. [12]. And, Rother et al.
[24] proposed an interactive graph-based method that uses
some global features of images iteratively. There are many
studies in image segmentation using graph based algorithms.
III. MATERIALS AND METHODS
We used the methods and the algorithm explained below
and tested our method on the color images from Single
Object Segmentation Evaluation Database [25]. Before we
explain our method, we should review some background
information about the materials and methods.
A. Graph-based Image Segmentation
In the constructed graph from in an image,
the vertices or nodes set represent the
pixels and weighted undirected edges, which have a positive
distance value and have no certain direction, represent the
similarities between adjacent pixels as shown Fig. 1 and Fig.
2. The weight of each edge is computed with some different
measures. Euclidean distance (Eq. 2) is one of the distance
measures used for the weight of the edges [26].
A graph built from pixels of an image can be represented
different pixel neighborhoods such as 4-neighborhood and 8-
neighborhood. Neighborhood-based clustering algorithm
proposed by Zhou et al. performs good data clustering,
because it can discover cluster of similar data and borders
using neighboring relationship between data points [18]. In
our method, we used 8-neighborhood mapping.
We used Euclidean distance for measurement of edges
weights in RGB (Red, Green and Blue) color space as in Fig.
2. To find the Euclidean distance between two adjacent
pixels, we applied the Eq. 2 [27].
(2)

Figure 2. Graph representation of the color pixels w
with the edge weights
between their neighboring pixels
B. Boruvkas Minimum Spanning Tree Algoorithm
A spanning tree is a sub graph of a grraph . contains
every vertex and does not contain any cyccles in the graph.
A graph might have more than one spaanning tree. The
undirected edges in a spanning tree can hhave also weight
values. The tree that has the least total weight is called
Minimum Spanning Tree (MST). Some grreedy algorithms
such as Prims Algorithm [28], Kruskals allgorithm [29] and
Boruvkas algorithm [30] are popular to finnd the MST in a
graph. In these algorithms, Boruvkas algorithm is also
unconcerned with any sequential process and orders
[8,31,32]. Each node or sub tree act inddependently from
others. In this study, Boruvkas algorithm is used because it
creates not much redundant small segmennts and need not
much preprocessing such as blurring filtter when looked
segmentation results according to [33]. Borruvkas algorithm
is also known as Sollins algorithm. It foorms a minimum
spanning tree by running the following stepss.
Step 1: Each vertex is considered to be subset and has a
subset number different from others.
Step 2: Each subset selects the minimum m-weighted edge
according to Euclidean distance, which connnects it to one of
the neighboring subsets as shown in Fig. 3.
Step 3: If subsets at both ends of the seleected edges dont
have the same subset number, these subsetss are merged and
have the same subset number in any sequencce.
Step 4: If there is only one subset,, the process is
terminated. Otherwise it continues from stepp 2.
Figure 3. Representation of Boruvkas algorithm off the graph in Fig. 2
88
C. Felzenszwalb and Huttenlochers Segmentation
Criterion
Felzenszwalb and Huttenlocher [21] presented a merging
criterion in Eq. 3 that make use of o some local and global
differences of the two sub trees and the differences inside the
both sub trees that are adjaceent each other [10,12,
17,33,34,35]. The internal differencce of the sub tree is the
highest edge weight in the sub s tree. The differences
is the weight value of thet smallest edge linking
the two adjacent sub trees and . refers to the size of
the sub tree , to explain, total verttices numbers in it, and k
refers to a constant. If satisfies the merging
criterion seen in Eq. 3, the two su ub trees will be merged.
Otherwise, it will pass without any operation.
o
  (3)
D. Implementation of the Methods
Firstly, we draw a rectangle seen in the Fig. 4 or
rectangles which we use to take th he intended object to be
entirely in. To look at the overall process, the steps of the
segmentation method can be summaarized in Fig. 5.
The method can be explained as a continuing. We got the
image into the 3D array image[M][N][3] in which 3 is the
color number of each pixel in RGB B color space. And then,
we prepared the 2D array nodes[[M][N] that consists of
objects of the Node class. We progrram the method by taking
advantage of Object Oriented Prograamming (OOP) and using
the C++ programming language.
Figure 4. Taking the intended object into an area
A Node object derived from the Node class. It includes
some information about the pixel reepresented. It includes the
cluster number that indicates whichh cluster it belongs to, the
RGB color values, the pointer roo ot that addresses its root
node, the information whether the pixel
p is in the drawn area
or not, the set of neighboring nodes that keeps the pointers of
the neighboring nodes accorrding to 8-connected
neighborhood, and the set of conn nected nodes. In addition
s , , the nearest
these, root nodes keep the cluster size
node outside the cluster, against node
n of the nearest node
inside the cluster, and the distance value  Through the recursive function, some sequential
between these nodes in other words between the clusters
and which each of the two nodes belongs to. Initially, all
the nodes have different cluster number, namely, each node
is a cluster at the same time, and each node is a root node.
Moreover, Node objects have some functions that run
recursively by linking the connected nodes and they are
doing in the same way too. By the way, we can reach whole
region and execute the intended processes with a node. To
give an example these functions, there are functions of
adding neighbor, connecting to a node, changing the cluster
number and the pointer linking the root node, getting the
nearest node of the nearest region, and so on.
Figure 5. Flowing chart of the segmentation and the object extraction
steps
After initializing the array nodes[M][N], we set the edges
between the neighboring nodes and their weight according to
Euclidean distance by using the RGB color values of the
pixels. After then, we used a self-recursive function that
composes trees by using Boruvkas algorithms with
Felzenszwalb and Huttenlochers comparison criterion. With
this criterion, it appears several trees.
89
operations are performed. Firstly, it finds the root nodes for
each cluster and computes that refer the number of roots in
whole present graph so that the further loops in the function
will be cycled until this number. At the following step in the
function, the process of finding the nearest region of each
region is executed. The information about that is kept in the
root nodes. And, it started to merge regions according to the
criterion through the cycles. So, the process of the function is
completed. If there is at least one merging at the last process,
the function will be recalled by itself. Otherwise, the
segmentation will be done.
Finally, to extract the intended object, all the neighboring
segments inside the bounded area are merged with each
other, and other neighboring segments are also merged in the
same way as seen Fig. 6. Showing the final clusters and the
object on the screen can be performed by using the root
nodes. For deletion small segments inside the area, the last
process can be performed by the way that the list of the
nearest neighboring region pairs is sorted according to the
distance  in ascending order and allowed to
merge region those are lower than the given parameter of the
limit value of the least cluster size that we have determined.
Figure 6. Representation of a merging according to case of selected area
F-score (F1 score or F-measure) is one of the
measurements of an image segmentation accuracy [36]. It
takes into account both the precision value and the recall
value to compute the score. The precision is the ratio of the
number of correct positive results of the extracted region to
the number of all intended region (human segmentation), and
the recall is the ratio of the number of correct positive results
of the extracted region to the number of all extracted region.
F-score reaches its best value at 1 and worst at 0 [37]. In Eq.
4, it is shown how to compute the F-score traditionally. The
formula is called traditional F-measure or balanced F-score
and is achieved by calculating the harmonic mean of the
values of precision and recall.
(4)
IV. EXPERIMENTAL RESULTS
We applied the method to several images from Single
Object Segmentation Evaluation Database [25]. The inputs
and the outputs of some of them are shown in Fig. 7. We
used the value 300 for the constant for these examples.
 Finally, to compare our methodd with other segmentation
methods those have been perform med on the database, we
perform our method overall sing gle object color image
database (100 images). We obtain n the scores of the other
(a) base [25].
methods from the source of the datab

TABLE II. F-SCORE TABLE OF OVER

RALL SINGLE OBJECT IMAGE
DATABASE

Method ore
F-Sco Remark
Our method 0.88 0.0
014 Interactive
Unified approach [36] 0.87 0.0
010 Interactive

(b) Probabilistic bottom-up [38] 0.86 0.0

012 Automatic
Texture segmentation [39] 0.83 0.0
016 Automatic
Normalized cuts [40] 0.72 0.0
018 Automatic
Mean shift [41] 0.57 0.0
023 Automatic

Because of the pre-intervention tot draw intended area, we

assume our method as interactive. To look the scores in
(c) Table II, it seems that our score is the best among the other
scores.
We coded the program with C++ + programming language.
To import and illustrate the images, we used OpenCV (Open
Source Computer Vision) library [42 2].
V. CONCLU
USION

(d) Image segmentation process is ana important intermediate

level of computer vision processes that is used widely in
many field. Object extraction is an important stage of some
of these fields. In this paper we app
plied a graph-based image
segmentation method by benefiting from literature. We tried
to show how to extract an object froom a segmented image by
using our object extraction method and
a we demonstrated that
(e) our method is how efficient. Anotther characteristic of the
method is that it is fast becausse of making a greedy
algorithm and using only color diffferences and cluster size.
As the operations that we showed are examined, it can be
Figure 7. Examples of segmentation results and the nnames of the images understood that the object extractiion method also can be
are (a) 100_0109, (b) beltaine_4_bg_0500502, (c) used upon another graph-based imaage segmentation method.
yokohm060409_dyjsn191, (d) tendrils and (e) pic0203vn0092 This shows that the success of the process also depends on
the segmentation method to be used.
We tested the images with the evaluaating function F-
score. The results for the images in Fig. 7 are as shown in ACKNOWLEDGEM
MENT
Table I. This study was supported by Selcuk University OYP
YP-094).
Coordination (Project No. 2015-OY
TABLE I. GES IN FIG. 7
F-SCORE TABLE OF THE IMAG
Image Name Precision Recaall F-Score
100_0109 0.95 0.933 0.94 REFERENC
CES
beltaine_4_bg_050502 0.84 0.999 0.91
yokohm060409_dyjsn191 0.92 0.966 0.94
tendrils 0.84 0.822 0.78 [1] G. Durusoy, Graph Cuts And Reegion Growing Algorithms in
pic0203vn0092 0.95 0.999 0.97 Medical Image Segmentation, The Po olytechnic Institute, New York
University, 2014
[2] W. Tao, H. Jin and Y. Zhang, Color Image Segmentation Based on
The results can vary because of the charracteristics of the Mean Shift and Normalized Cuts, IE EEE Transactions on Systems,
mage tendrils is
images. For example, the test result of the im Man, And CyberneticsPart B: Cyberrnetics, Vol:37 (5), 2007
worse than the others. Because, the object and background [3] Q. Huang, X. Bai, Y. Li, L. Jin and X.
X Li, Optimized graph-based
colors are very similar to each other. segmentation for ultrasound images, Neurocomputing 129, pp.216
224, 2014

90
[4] M. Grundmann, V. Kwatra, M. Han and I. Essa, Efficient
Hierarchical Graph-Based Video Segmentation, IEEE CVPR, pp.
2141-2148, 2010
[5] D. F. Fitzgerald, D. S. Wills and L. M. Wills, Real-time, parallel
segmentation of high-resolution images on multi-core platforms, J
Real-Time Image Proc, DOI 10.1007/s11554-014-0432-z, 2014
[6] Y. Deng, B. S. Manjunath and H. Shin, Color image segmentation,
Proc. IEEE Computer Society Conference on Computer Vision and
Pattern Recognition CVPR'99, Fort Collins, CO, volume: 2, pp. 446-
51, 1999
[7] N. Y. Bayramoglu and C. F. Bazlamacci, Recursive Shortest
Spanning Tree Algorithms for Image Segmentation, EUROCON
2005 Belgrade, 2005
[8] A. N. Skurikhin, Hierarchical Image Feature Extraction by an
Irregular Pyramid of Polygonal Partitions, ASPRS 2009 Annual
Conference, Baltimore, Maryland, 2009
[9] J. Ding, R. Ma and S. Chen, A Scale-Based Connected Coherence
Tree Algorithm for Image Segmentation, IEEE Transactions on
Image Processing, Vol:17 (2), 2008
[10] B. Peng, L. Zhang and D. Zhang, A survey of graph theoretical
approaches to image segmentation, Pattern Recognition
46(2013)10201038, 2012
[11] J.A. Bondy, U.S.R. Murthy, Graph Theory with Applications,
American Elsevier Publishing Co., U.S.A., 1976
[12] P. F. Felzenszwalb and D. P. Huttenlocher, Efficient graph-based
image segmentation, International Journal of Computer Vision59
(2), pp.167181, 2004
[13] Y. Xu, V. Olman and D. Xu., Minimum spanning trees for gene
expression data clustering, Genome Informatics, Vol:12, pp.2433,
2001
[14] S.H. Kwok and A.G. Constantinides, A fast recursive shortest
spanning tree for image segmentation and edge detection, IEEE
Transactionson Image Processing, Vol:6 (2), pp.328332, 1997
[15] S.H. Kwok, A.G. Constantinides and W.C. Siu, An Efficient
Recursive Shortest Spanning Tree Algorithm using Linking
Properties, IEEE Transactions on Circuits And Systems for Video
Technology, Vol: 14 (6), June 2004
[16] C. T. Zahn, Graph-Theoretical Methods for Detecting and
Describing Gestalt Clusters, IEEE Transactions On Computers,
Vol:20 (1), pp.68-86, 1971
[17] O. Grygorash, Y. Zhou and Z. Jorgensen, Minimum Spanning Tree
Based Clustering AlgorithmsProceedings of the 18th IEEE
International Conference on Tools with Artificial Intelligence
(ICTAI'06) 0-7695-2728-0/06 $20.00, 2006
[18] S. J. Peter, Minimum Spanning Tree-based Structural Similarity
Clustering for Image Mining with Local Region Outliers,
International Journal of Computer Applications (0975 8887) Vol:8
(6), 2010
[19] D. Lopresti, J. Zhou, Locating and recognizing text in www
images, Information Retrieval, pp.177206, 2000
[20] O.J. Morris, M. de J. Lee, A.G. Constantinides, Graph theory for
image analysis: an approach based on the shortest spanning tree, IEE
Proceeding F, Communications, Radar and Signal Processing
Vol:133, pp.146152, 1986
[21] P. F. Felzenszwalb and D. P. Huttenlocher, Image Segmentation
Using Local Variation, In Proceedings of IEEE Conference on
CVPR, p:98104, 1998
[22] Y. Haxhimusa and W. Kropatsch, Segmentation Graph Hierarchies,
Proceedings of Joint Workshops on Structural, Syntactic, and
Statistical Pattern Recognition S+SSPR. Vol:3138, Lecture Notes in
Computer Science pp.343-351, 2004
[23] J. Wassenberg, W. Middelmann and P. Sanders, An Efficient
Parallel Algorithm for Graph-Based Image Segmentation, Computer
Analysis of Images and Patterns, LNCS Vol:5702 pp.10031010,
2009
91
[24] Carsten Rother, Vladimir Kolmogorov, Andrew Blake, GrabCut:
interactive foreground extraction using iterated graph cuts, ACM
Transactions on Graphics (TOG), Volume 23 Issue 3, 2004
[25] http://www.wisdom.weizmann.ac.il/~vision/Seg_Evaluation_DB/inde
x.html (Accessed in 2015)
[26] F. Malmberg, Graph-based Methods for Interactive Image
Segmentation, Acta Universitatis Upsaliensis Uppsala ISSN: 1651-
6214 ISBN: 978-91-554-8037-0, 2011
[27] H.D. Cheng, X.H. Jiang, Y. Sun and J. Wang, Color image
segmentation: advances and prospects, Pattern Recognition 34,
2259-2281, 2001
[28] R. Prim, Shortest connection networks and some generalization,
Bell Systems Technical Journal, Vol:36, pp.13891401, 1957
[29] J.B. Kruskal, On the shortest spanning subtree of a graph and the
traveling salesman problem, Proceedings of the American
Mathematical Society 7 (1): 4850, JSTOR 2033241, 1956
[30] O. Boruvka, O jistem problmu minimalnim (About a certain
minimal problem), Prace mor. prirodoved. spol. v Brne III (in
Czech, German summary) 3: 3758, 1926
[31] Y. Haxhimusa, A. Ion, W. G. Kropatsch and T. Illetschko,
Evaluating Minimum Spanning Tree Based Segmentation
Algorithms, Computer Analysis of Images and Patterns Volume
3691 of the series Lecture Notes in Computer Science, pp.579-586,
2005
[32] S. Chung and A. Condon, Parallel Implementation of Borikkas
Minimum Spanning Tree Algorithm, IEEE Proceedings of IPPS 96,
1996
[33] M. Tepper, M. Mejail, P. Mus and A. Almansa, Boruvka Meets
Nearest Neighbors, HAL Id: hal-00583120, 2011
[34] S. C. Turaga, Learning image segmentation and hierarchies by
learning ultrametric distances, Massachubetts Institute of
Technology, 2009
[35] A. Fahad and T. Morris, A Faster Graph-Based Segmentation
Algorithm with Statistical Region Merge, ISVC'06 Proceedings of
the Second international conference on Advances in Visual
Computing - Volume Part II Pages 286-293 Springer-Verlag Berlin,
Heidelberg, 2006
[36] S. Bagon, O. Boiman and M. Irani, What is a Good Image Segment?
A Unified Approach to Segment Extraction, ECCV 2008 pp.30-44,
2008
[37] C. J. van Rijsbergen, Book Information Retrieval (2nd ed.),
Butterworth-Heinemann Newton, MA, USA, 1979
[38] S. Alpert, M. Galun, R. Basri and A. Brandt, Image segmentation by
probabilistic bottomup aggregation and cue integration, in:
International Conference on Computer Vision and Pattern
Recognition, pp. 18, 2007
[39] M. Galun, E. Sharon, R. Basri and A. Brandt, Texture segmentation
by multiscale aggregation of filter responses and shape Computer
Vision, pp. 716723, 2008
[40] J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE
Transactions on Pattern Analysis and Machine Intelligence 22 (8) pp.
888905, 2002
[41] D. Comaniciu and P. Meer, Mean shift: a robust approach toward
feature space analysis, IEEE Transactions on Pattern Analysis and
Machine Intelligence 24 (5), pp. 603619, 2002
[42] http://opencv.org (Accessed in 2015)