2011 International Conference on Electronic Devices, Systems and Applications (lCEDSA)
Fruit defect detection from color images using ACM
and MFCM algorithms
Ghobad Moradi Mousa Shamsi
Member of Young Faculty of Electrical
Researcher Club, Branch Engineering, Sahand
of Kermanshah Azad University of Technology
University, Iran, Tabriz, Iran
ghobad.moradi(@yahoo.com shamsi(@sut.ac.ir
Ab stract-Segmentation of an image into its components plays an
important role in most of the image processing applications. In
this article an important application of image processing in
determination of fruit quality is studied, and an automatic
algorithm is proposed in order to determine fruits skin color
defects. Removing of image background and extraction of fruit
shape, exactly, at presence of shadow and complex background is
considered as an important preprocessing stage. In proposed
algorithm at first, background in image is omitted by using active
counter model (ACM) algorithm. Finally, the image is segmented
using modified FCM (MFCM) algorithm. Experimental results
on fruit color images show that proposed algorithm increases
accuracy and speed of fruit skin defect detection, considerably.
Keywords-defect detection; fruit color image; image
segmentation; fuzzy c-means; active contour model
I. INTRODUCTION
Increasing customer knowledge about product quality leads
to competitive fruits and vegetable trade. High quality is the
main reason for survival of fruits commerce. Always,
producing high quality products and using image processing in
investigation of its quality are important [1, 2]. In traditional
reliable quality method, human operators are used. It is boring
and time consuming [3, 4]. For example most of the food
products are clustered by seasonal workers. Work shift is
sometimes long and working condition is hard and mostly there
is time limitation for reliability of products freshness.
Clustering is a repetitive and boring affair needs to long term
periods [5]. So, it may effects the products quality
performance. Thus, machine vision is used for quality
evaluation and sorting of fruits like apple based on quality.
Since it will increase speed and also decrease human error in
sorting procedure. In [6] a Gaussian model has been presented
for segmentation and extraction of Golden apple defects from
color images. Also, in [7] Bayesian clustering method is used
to process Jonagold apple color images. In [8] an acquired
controlling system and different clustering methods is used for
detection of cranny smith apple defects. In [9] a flooding
algorithm has been used for segmentation of fruit defects on
gray images. This method is applicable in products with
uniform color skin, the improved version of this method has
been proposed in [10] that by employing ACM algorithm can
determine defect area, exactly. In [11] a neural network was
978-1-61284-389-6/11/$26.00 2011 IEEE
Mohammad H. Sedaghi Mohammad R. Alsharif
Faculty of Electrical Faculty of Engineering,
Engineering, Sahand University of the Ryukyus
University of Technology Okinawa, Japan
Tabriz, Iran asharif@ie.u-ryukyu.ac.ip
sedaaghi@sut.ac.ir
proposed for segmentation of defect in Jonagold apples, and
segmentation was highly accurate but sepal and tail areas are
considered defect most of the times. Linear discrimination
analysis for segmentation of an image into three or four classes
was proposed in [12] that discriminator function arranges
apples as accepted or rejected. The accuracy of this method
was high for apples. In [13], Fourier analysis on apples was
used for study of quality. The method showed that human
perception of quality affected by products external features. In
presence of several and complex features, human error is
increased in product clustering. Ultra spectral imaging
technology for inspection and grading of agricultural products
and multispectral images for different defects recognition were
used in [14] and [15], respectively. In our previous works [16-
18], we have presented automatic algorithms for segmentation
of the internal structures of pomegranate and external
structures of apple. In [19, 20] an algorithm was proposed for
detection and removing of shadow in fruits color images in
HSV color space by using homomorphic filter that employing
intensity component in HSV color space relative to gray
images and red component in RGB color space indicates the
best results in omitting images shadow. This article is
organized as follows: In section 2, the proposed algorithm is
discussed. In section 3 experimental results and in section 4
conclusions are proposed.
II. PROPOSED ALGORITHM
Figure 1 shows flowchart of the proposed algorithm: In
order to recognize fruit skin defects by the minimum error, at
first redundant data like shadow and background are omitted.
For this purpose, by selecting proper color space, ACM
algorithm is used for omission of shadow and background. So,
we extract only fruit shape from image. For minimizing error,
extracted fruit images are entered to the clustering algorithm as
input. In this paper, a modified fuzzy means (MFCM) method
is used for fruit skin defects segmentation.
182

Figure I. Proposed algorithm.
A. Color Space
In this paper, on wide range of data, ROB, rgb, HSV, La*b*
and YCbCr color spaces were tested for fruits image
processing and it was concluded that every rgb, Lab and
YCbCr color spaces can be a proper color space for feature
extraction and image segmentation under different varying
illumination. In this paper, the La*b* color space has been
used for preprocessing and segmentation steps.
B. Fruit shape extraction using ACM algorithm
In order to automatic fruit shape extraction in presence of
factors like shadow, in this paper, an ACM method is proposed
in proper color space. By using this model the fruit shape is
extracted from image, exactly. This model is an optimization
method to minimize energy that controlled by external, internal
and image forces that converges the contour towards features
like lines and edges. Among the earliest and most well known
deformable models is the ACM known as snakes proposed by
Kass [22]. A snake is an energy minimizing parametric contour
that deforms over a series of time steps. Each element along the
contour u depends on two parameters, where the space
parameter s is taken to vary between 0 and N-J, and t is time
(iteration):
U(s,t) = (x(s,t),y(s,t)).
The total energy Esnake of the model is given by the sum of the
energy for the individual snake elements:
J,N-l
Esnake =
0 Eelement (u (s, t)) ds.
The energy for each element can be decomposed into three
basic energy terms:
Eelement Eint(u) Eext(u) Eimage (u) .
= + + image [23].
In ACM, every point on contour moves to an adjacent pixel
until minimize Esnake in each step of repetition process
(Fig. 2). The process stops when a local minimum is met.
The internal energy (Einternal) incorporates regularizing
constraints that give the model tension and stitfuess. The
external energy (Eexternal ) represents external constraints
imposed by high-level sources such as human operators or
automatic initialization procedures. The image (potential)
energy ( Eimage) drives the model towards salient features; it is
usually generated by processing the image to enhance light
and dark lines, edges, or terminations [23]. The internal
energy of a snake element is defined as:
Einternal(U) =
a (s) 1:12 + (s) I:I. (4)
e f
Figure 2. Energy minimization in ACM algorithm.
This energy contains a first-order term controlled by a(s),
and a second-order term controlled by (s). Minimizing the
first-order energy term makes the snake contract by
introducing tension. Minimizing the second-order term makes
the snake resist bending by producing stiffuess. In other
words, the curve is predisposed to have minimal (preferably
zero) velocity and acceleration with respect to the parameter s.
The weights a(s) and (s) control the relative importance of
the tension and stitfuess terms.
Both automatic and manual supervision can be used to
control external forces that drive ACMs toward or away from
specific feature [23].
A potential (image) energy P generated by processing an
(1)
image J(x, y) is used to drive snakes towards features of
interest. The potential energy terms drive snakes towards lines
(regions), edges and detecting termination (comers). The total
image energy can be expressed as a weighted combination of
these functional:
(2)
The image energy is thus a linear combination of line, edge
and termination energy terms, all computed from the raw
(3)
C. Modified FCM algorithm
In FCM, fuzzy concepts and membership function are used
for labeling each pixel to classes. In other words, the
probability of belongs of a pixel to classes is calculated in each
repetition and at the end of the segmentation a pixel is labeled
to a class that has the highest membership. So, the aim of FCM
183
is minimization target or error function in image segmentation
trend that it is defmed as follows in fuzzy model [24].
In above relation Ilik is the probability of belongs of k th pixel
to i th class, Ilxk - vil12 is distance of k th pixel from i th class
center, nis total image pixels, and c is the number of classes. In
this relation, parameter m is fuzzy coefficient that it is a real
number greater than one. It is clear that since !1ik is a
probability function, the following condition will be necessary.
,i = 1,2, ... ,N.
the algorithm result on image with complex background. As
you seem ACM algorithm is optimal relative to images by
simple or complex background and omit background, exactly.
(6) The proposed algorithm results are shown in figure 4. In
this figure, column (a) shows original color images. Column
(b) shows the results without applying ACM algorithm. The
number of classes is 3. As it is observable, some part of defect
and background are labeled in the same class. So, in order to
improve segmentation algorithm result, first background and
shadow are removed by ACM algorithm. The result of ACM
algorithm is shown in column (c). Output of this stage is
utilized as input of the MFCM algorithm. The results are
(7) shown in figure 4(d).

A necessary condition for minimization of error function IV. CONCLUSION

(equation (6)) is that its gradient with respect to Jlik be zero.
Thus for doing so and updating, the probability of a pixel
membership in a class in each repetition of algorithm is
calculated as follows {K.. XTjuavy, 2006 #196}:
-1
u t+ (1IXk-Vill)2/Cm_l)
l - Jlik - L.. j=l IIx
- [""C k-vjll
Where 1 ::; i ::; c and 1 ::; k ::; N , also classes centers are
updated as follows. Error function minimization algorithm is
continued when there is no II Ut+1 - UtII ::; E.
In this paper, an automatic algorithm was proposed in
order to detect fruit skin defect based on fruits color image.
Removing of background in presence of shadow by ACM
algorithm lead to increase algorithm performance regarding
accuracy and speed in segmentation of defects. In our
] (8) proposed method, instead of using all image pixels in
segmentation algorithm, only pixels of fruit shape were
utilized. In our future work, it is tried to present an algorithm
for evaluation of quality and extraction of external and internal
defects using both fruit color images and MR images.

(9)

One of the important characteristics of image is that the

value of a feature in neighbor pixels is similar and the
probability that they belong to a class is high. But in FCM
algorithm, this characteristic has not been used. So, in this
paper MFCM optimized method is proposed as follows that
information about adjacent pixels is used for increasing
clustering accuracy. Modified algorithm is FCM algorithm
with this difference that its membership function is updated as
follow:

Ilijsij
Jlij =
"c m n . (10)
"'k=lllij Sij

Where Sij Lk=NCXj) Jlik is called spatial function and

=

N (Xj) is a square window with central pixel Xj in spatial

domain. Spatial function sij shows the probability that pixel Xj
belongs to i th class. Spatial function of a pixel for a class is
high when the majority of its neighbors belong to the same
class. In this paper a 3 x 3 window was used, meanwhile
parameters nand m is equal one.
III. EXPERIMENTAL RESULTS a b c d

The main propose of this paper is to extract fruit external Figure 3. ACM algorithm and fruit shape extraction: first raw: original color
defects from color images. So, the proposed method was images with (a) shadow (b) ,(c) simple background (d) complex background.
employed on fruit color images by simple and complex Second to seventh raw: ACM algorithm results. Last raw: extracted fruit
shape.
backgrounds. Figure 3 shows different stages of ACM
algorithm on color fruit images. In this figure, column (a)
shows the ACM algorithm in presence of shadow. The result
of algorithm on color images with simple background has
been shown in columns (b) and (c). Also, column (d) shows

184
 and Electronics in Agriculture, vol. 23, pp. 43-53,
1999.
[8] G. Rennick, Y. Attikiouzel, and A. Zaknich,
"Machine grading and blemish detection in apples, "
In Proc. 5th Int. Symp. Signal Processing and Appl,
Brisbane, Australia, pp. 567-570, 1999.
[9] Q. Yang, "Approach to apple surface feature
detection by machine vision, " Computers and
Electronics in Agriculture, vol. 11, pp. 249-264,
1994.
[10] Q. Yang and 1. A. Marchant, "Accurate blemish
detection with active contour models, " Computers
and Electronics in Agriculture, vol. 14, pp. 77-89,
1995.
[11] D. Unay and B. Gosselin, "A quality sorting method
for 'jonagold' apples, " In Proc. Int. Agricultural
Engineering Conf. (AgEng), Leuven, Belgium, 2004.
[12] E. Molto, "Multispectral inspection of citrus in real
time using machine vision and digital processors, "
Computers and Electronics in Agriculture, vol. 33,
pp. 121-137, 2002.
[13] Paulus, "Inspection and grading of agricultural and
food products by computer VISIOn systems, "
a b c d
Computers and Electronics in Agriculture, vol. 36,
pp. 193-213, 1997.
Figure 4. Segmentation algorithm results: a) original color images. b) [14] C. Yud-Ren, C. Kuanglin, and K. Moons, "Machine
Results without background removing. c) ACM algorithm results. d) Results vision technology for agricultural applications, "
after background removing.
Computers and Electronics in Agriculture, vol. 36,
pp. 173-191, 2002.
REFERENCES
[15] D. Ariana, D. E. Guyer, and B. Shrestha, "Integrating
multispectral reflectance and fluorescence imaging
for defect detection on apples, " Computers and
[1] Abbott and A. Judith, "Quality measurement of fruits Electronics in Agriculture, vol. 50, pp. 148-161,
and vegetables, " Postharvest Biology and 2006.
Technology, vol. 15, pp. 207-225, 1999. [16] G. Moradi, M. Shamsi, M. H. Sedaaghi, and M. R.
[2] P. Chen, "Quality Evaluation Technology for Alsharif, "Segmentation of pomegranate MR images
Agricultural Products, " International Conference on using ACM and FCM algorithms, " presented at the
Agricultural Machinery Engineering, 1996. International Conference on Graphic and Image
[3] C. Valero and M. R. Altisent, "Design Guideline for Processing (ICGIP), 2010.
Quality Assessment of Fresh Fruits in Hypermarket, " [17] G. Moradi, M. Shamsi, M. H. Sedaaghi, and M. R.
CIGR Journal of Scientific Research and Alsharif, "Segmentation of pomegranate MR images
Development, vol. 2, 2000. using spatial fuzzy c-means (SFCM) algorithm, "
[4] Y. C. Hung, S. E. Prussia, and G. o. Ezeike, presented at the International Conference on Graphic
"Nondestructive firmness sensing using a laser air and Image Processing (ICGIP), 2010.
puff detector, " Postharvest Biology and Technology, [18] G. Moradi, M. Shamsi, M. H. Sedaaghi, and M. R.
vol. 16, pp. 15-25, 1999. Alsharif, "Apple defect detection using statistical
[5] C .Puchalski, 1. Gorzelany, G. Zagula, and G. histogram based EM algorithm, " 19th Iranian
Brusewitz, "image analysis for apple defect Conference on Electrical Engineering, 2011,In press.
detection, " TEKA Kom. Mot. Energ. Roln, OL PAN, [19] M.R. Asharif and H. Etemadnia, "Automatic Image
vol. 8, pp. 197-205, 2008. Shadow Identification using LPF in Homomorphic
[6] V. Leemans, H. Magein, and M. F. Destain, "Defect Processing System, " in Proceedings of the VII-th
segmentation on 'golden delicious' apples by using Biennial Australian Pattern recognition Society
colour machine vision, " Computers and Electronics Conference, Digital Image Computing: Techniques
in Agriculture, vol. 20(2), pp. 117-130, 1998. and Applications, Macquarie University, Sydney,
[7] V. Leemans, H. Magein, and M. F. Destain, "Defect Australia, , vol. 1, pp. 429-437, 2003.
segmentation on 'Jonagold' apples using colour vision [20] M.R. Asharif and H. Etemadnia, "Homomorphic
and a Bayesian classification method, " Computers Processing Approach for Image Shadow

185
 Identification,"International Symposuim on
Telecommunications, Isfehan, Iran, 2003.
[21] H. Good, "Measurement of color in cereal products,"
Cereal Foods World, vol. 47, pp. 5-6, 2002.
[22] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes:
Active contour models," International Journal of
Computer Vision, vol. 1,pp. 321-331,1988.
[23] M. Nixon and A. Aguado, feature extraction and
image processing, illustrated ed.: Newnes, 2002.
[24] K. S. Chuang,H. L. Tzeng, S. Chen, J. Wu, and T. J.
Chen., "Computerized Medical Imaging and
Graphics," Fuzzy c-means clustering with spatial
information for image segmentation, vol. 30, pp. 9-
15,2006.

186