Computers and Electronics in Agriculture

31 (2001) 1729
Measuring image analysis attributes and
modelling fuzzy consumer aspects for tomato
quality grading
Gerhard Jahns
a,
*, Henrik Mller Nielsen
b
, Wolfgang Paul
a
a
Institute of Biosystems Engineering, Bundesallee 50, D-38116 Braunschweig, Germany
b
Department of Agricultural Sciences, Agro6ej 10, DK-2630 Taastrup, Denmark
Abstract
Quality grading is done by the consumer in a fuzzy way according to his senses sight,
touch and smell. Visual appearance is the main source of information and can be broken
down by image analysis to attributes like size, colour, shape, defects and abnormalities.
Moreover, these attributes are often correlated with nutritional or other sensual quality
parameters. Starting with such basic quality attributes, with the example of tomatoes a
reasoning is proposed, mapping various fuzzy consumer aspects to overall quality classes.
The objective is to achieve an automatic rating of fruit quality, modelling consumer aspects
and producer needs. Such a mapping of fuzzy image analysis attributes to an overall visual
quality reduces destroying tests. The reasoning can easily be rearranged and optimised
according to varying consumers expectations. 2001 Elsevier Science B.V. All rights
reserved.
Keywords: Image processing; Fuzzy reasoning; Tomato quality
www.elsevier.com/locate/compag
1. Introduction
Ofcial quality denitions for fruit or vegetables are hardly more than a rough
rating on size and colour. Where the USDA grade standard for tomatoes (USDA,
1991) species six maturity stages based on the dominating colour of the tomato,
the EU standard as described by (AID, 1992) effectively has two stages: green and
* Corresponding author. Tel.: +49-531-596466; fax: +49-531-596369.
E-mail address: gerhard.jahns@fal.de (G. Jahns).
0168-1699/01/$ - see front matter 2001 Elsevier Science B.V. All rights reserved.
PII: S0168- 1699( 00) 00171- X
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 18
Table 1
Quality items as seen by the consumer
Size, colour, defects, maturity Sight
Firmness, grip, maturity Touch
Juiciness, sweetness, sourness, aroma Smell/taste
Hope for vitamins, nutrients, valuable elements Nutrients
Fear of plant protection agents, nitrate, natural toxins Toxins
red (a Dutch auction colour scale covers 12 colour classes). No objective measures
of shape are given, whereas size grading is done by measuring the largest diameter:
3035, 3540, 4047, 4757, 5767, 6782, 82102 and 102 mm and more. The
size tolerance for each interval is 2 mm. Tomatoes are divided into three classes
(class Extra, class I, and class II) based upon the overall state of the tomato
(rmness, number and extend of defects or blemishes), but most of these state
variables are not specied in an objective manner. With respect to uniformity:
tomatoes from the classes Extra and I should have the same colour and maturity.
So far with ofcial quality denitions. Clearly, the consumer is looking for more.
The produce should be appealing by sensual check (sight, touch, organoleptic, etc.),
nutritional values should be satised and security factors (residues of fertiliser or
plant protection agents as well as natural toxins) must be met. Thus Table 1 would
be the ideal denition from the consumers point of view. One has to admit, that
Table 1 contains several fuzzy denitions, items set up irrespective whether the
quality characteristics are measurable or not. So the ideal consumer quality has to
be translated into practice. The quality items in Table 1 have to be translated into
easily measurable factors.
Table 2 is a breakdown of Table 1 relating physical quality parameters to what
interests the consumer or the grower. Measurements according to Table 2 would be
a full scale quality assessment, denitely not manageable in practical production.
But with these measurements, correlation between the above parameters can be
achieved. Table 3 is a possible reduction of Table 2, relating some of these quality
parameters to image analysis. It is shown later on, that attributes of appearance
correlate also with quality parameters like rmness, sugar/acid ratio or vitamin C.
Table 2
Physical parameters related to the needs of consumers and producers
Quality aspects Properties Sensorics
Area, spectral analysis, texture, Weight, maturity, shape, position, Image analysis
curvature greenback, surface defects
Penetrometer Elasticity, hardness skin, maturity E-Modulus, rupture
Taste intensity, aroma Human team Taste limit
Sugars, vitamin C, BRIX, TOC, Photometrics, Nutritionals, sugar/acid ratio specic ions
test strips NO
3
, K
+
, Ca
2+
pH-value, conductivity Electronics Acidity, total salts
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 19
Table 3
Quality parameters related to image analysis
Image analysis attributes Consumer aspects
Weight, Area
Spectral analysis Maturity, elasticity, sugar/acid ratio, greenback
Texture Surface defects
Curvature Shape
Finally, a fuzzy rating is recommended as a basis to process such a basic
quality parameter set. Here a reduction to image analysis attributes is proposed,
but other automatically measurable parameters are easily included. The main
objective is to reduce quality assessment to few characteristics, which can be
measured by non contact and non destroying tests, preferably by image process-
ing. Only a few and random destroying extra checks are necessary then for
counter checking.
2. Image analysis and quality attributes
A standard colour camera, frame grabber and PC is used for image analysis
to measure visible quality parameters. Illumination is done in a half sphere in
order to avoid shadows. Further descriptions may be found in Nielsen and Paul
(1995). Here only the results of practical quality assessments are summarised.
2.1. Size and size distribution
Size is taken here as (1) the largest diameter the major axis and (2) the
area of a tomato viewed from above. The minor axis and perimeter measure are
needed later for computing the shape indices in the following section. The area
is computed from the 2D image of a tomato and so an estimate of the tomatos
weight/volume is achieved (Fig. 1).
The highest degree of correlation was found between area measured by image
analysis and weight measured using a scale:
W=0.0021A
1.3614
, (R
2
=0.9955) (1)
where A is the area (mm
2
) and W is the weight (g) (Eq. (1)). Using the above
relationship, the mean absolute error of weighing a tomato by determining the
area using image analysis would be 2.06% (approximately 1.8 g for 90 g
tomato). Thus, size and size distribution or weight and weight distribution can
be recorded automatically with a high degree of accuracy by image analysis.
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 20
2.2. Colour and colour uniformity
The colour of a tomato is an accepted measure for its maturity stage (Sarkar and
Wolfe, 1985a,b; Choi et al., 1995). The procedure for estimating the maturity stage
of a tomato by measuring its colour using image analysis is done by placing the
tomato on a small plate having approximately the same luminance as the
tomatoes in the illumination chamber and an RGB image is acquired. Using
colour calibration data, the RGB image, I
RGB
(i, j ), is converted to a standardised
CIE/XYZ image (Nielsen and Paul, 1995, 1996).
The result of computing is either a normalised red/green index or better the
dominating wavelength as maturity index. For some 60 Pannovy tomatoes the
result of the colour discrimination is compared to human grading, as shown in Fig.
2. Overlapping is found at maturity stages 68, thus the human grouping of the
tomatoes is not perfect.
2.3. Shape and cur6ature
The global shape measures compactness and eccentricity is estimated from the
measured parameters area, perimeter, major and minor axis as shown in Fig. 3.
This provides a simple method of judging the overall shape of tomatoes.
In order to locate and determine the extent of local shape errors, an analysis of
the tomato boundary curvature is an efcient solution. The boundary curvature is
computed from the smoothed version of the contour data (Figs. 4 and 5).
This method clearly detects and locates concavities and convexities, also a not
sellable curvature.
Fig. 1. Relationship between area measured by image analysis (blossom end view) and weight measured
using a scale for tomatoes of different size and maturity.
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 21
Fig. 2. Dominating wavelength as maturity index.
2.4. Spots and scars
The colour distribution of a tomato is easily evaluated using, e.g. the dominating
wavelength. The histogram of the dominating wavelength of the pixels of a
discoloured tomato (Figs. 6 and 7) shows a clear threshold value between the
yellow spots and the red colours, thus providing a way of detecting colour
abnormalities. The histogram method is also easily capable of detecting greenback
tomatoes or tomatoes with bottom-end rot. Either the total spread of the distribu-
Fig. 3. Example of shape sorting based on the calculated compactness and eccentricity values: a
misshapen (C=0.9725, E=0.8683) and a perfectly shaped (C=0.99807, E=0.9929) tomato.
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 22
Fig. 4. Tomato with several local shape errors located by analysing the boundary curvature (Fig. 3).
tion or the attribute more than one peak can be taken as a measure for colour
abnormalities. Furthermore, scars or injuries can be detected with texture analysis
(differences in reection within neighbouring pixels) or black/white discrimination,
see Fig. 8. Surface injuries have as a result a discontinuity in reection.
2.5. Correlation between optical quality attributes and inner 6alues
Within different stages of maturity or different groups of constituents, values of
quality parameters sometimes develop in parallel. It is interesting to look at
correlation between these measurements. The aim is to reduce tedious measure-
ments, because when the fruits of one variety are coming from the same greenhouse
Fig. 5. The boundary curvature calculated for the tomato shown in Fig. 4.
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 23
Fig. 6. Unevenly coloured tomato (yellowish spots). The overlay shows the areas found by analysing the
histogram of dominating wavelengths.
with more or less the same nutrition and climate, quality assessment should be
reduced to non contact, non destructive tests.
The clearest correlation is between colour and rmness. Both measures have to
do with maturity. No doubt, maturity can easily and with good result be measured
with colour assessment (Fig. 9). Only a weak correlation exists between Brix and
maturity (Fig. 10). The same can be stated about maturity and vitamin C (Fig. 11).
Although there is a tendency, the correlation is not very good.
Fig. 7. Histogram (number of pixels) of the unevenly coloured tomato depicted in Fig. 6 over the
dominating wavelength of each pixel.
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 24
Fig. 8. Reection discontinuities in the red band of an over mature tomato with a scar.
3. Fuzzy fusion: quality attributes
For the overall objective of fuzzy mapping, a model for processing the visual
fuzzy parameters into some classes of an output parameter total visual quality is
proposed.
3.1. Basic quality attributes
Consumers as well as producers need some basic quality standard, which is
generally accepted and is measurable by devices reasonable in price. Producers can
assess the results of their management decisions, consumers are looking for better
quality. As a result from the above discussions attributes and classes listed in Fig.
12 and Table 4, for quality assessment are proposed. According to mans impres-
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 25
Fig. 9. Correlation between maturity and rmness.
sion, the proposed parameters are not crisp numbers, but distributions around a
given value. Here fuzzy triangular distributions are chosen with peak values of one.
The membership functions decrease to zero at the peak memberships of the
forgoing or following classes, as can be seen in the attached windows of Fig. 10 or
Fig. 11. Because the following fuzzy reasoning was done with a professional
software system, FUZZY CONTROL MANAGER (FCM) www.transfertech.de, varia-
tions like other membership functions (i.e. Gauss, etc.) could easily be applied.
3.2. Fuzzy rating of consumer aspects
Parameters for contour and shape as well as colour and colour uniformity are the
basic parameters to be processed. These four parameters belong to fuzzy classes
according to consumers judgements (Fig. 13). Contour and shape are processed by
fuzzy reasoning to an output called geometry, colour and colour distribution to an
Fig. 10. Correlation between maturity and BRIX value
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 26
Fig. 11. Correlation between maturity and vitamin C.
output impression. Both give the overall appearance, which had to be checked for
cracks and faults. The attribute cracks is more or less a knockout criterion. When
no discontinuities in colour or reection can be detected, a further grading to size
gives an overall visual quality as result.
The processing is done with classical fuzzy AND operators: IF (X1 AND X2)
THEN Y1. Also operators like GAMMA (compensatory AND) may be applied.
But these operators need a careful design. To ease plausibility control according to
human reasoning the whole system has been broken down to rule bases with two
inputs only. With such an approach the rule base is easily surveyed.
Fig. 14 gives an example of mapping the contour variables and shape variables
to appearance. The best appearance is with even contour and round shape. The
Fig. 12. Elements for automatic assessment of visual quality.
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 27
Table 4
Fuzzy variables and classes for quality assessment
Classes (triangular membership) Variable
Contour Deep incuts, light incuts, even, light swelling, severe deformation
Mishappen, lengthened, attened, round Shape
Green, breaker, turning, pink, mature red Colour
Even, light irregular, severe irregular, spots Colour uniformity
Cracks, no cracks Cracks
Excellent, good, medium, bad, very bad Appearance
Very small, small, medium, large, very large Size
Visual quality Not sellable, poor, medium, good, excellent
two-dimensional inuence mapping can easily be checked and tested due to
consumer intentions. The output membership distribution for constant shape and
parameters is also shown, defuzzication is straightforward.
Because of this stepwise tuning of parts of the model also the total visual quality
seems reasonable (Fig. 15). The shape and colour are mapped here to the fuzzy
quality output. The mountain of good quality gives an excellent feeling for
parameter inuences. The program also allows an optimisation of input parameters
for membership functions by back-propagation, if enough consumer interviews on
total quality are available.
4. Conclusion
Sorting and quality rating is normally done by experts. In consequence it is
subjective and the results show inter and intra individual variations. To achieve
objective and reproducible results, a quality assessment based on measured optical
characteristics and on a fuzzy model of consumer quality rating is proposed.
Optical measurements show a good correlation in respect to maturity parameters.
Fig. 13. Fuzzy tomato quality grading.
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 28
Fig. 14. Mapping of geometry on appearance.
Fig. 15. Mapping of colour and shape on quality.
G. Jahns et al. / Computers and Electronics in Agriculture 31 (2001) 1729 29
So contact or destructive measurements can be reduced to a minimum. The fuzzy
impression of consumers modelled by fuzzy logic can be ne tuned if enough data
are available. The proposed principle of modelling consumer quality can be adapted
and applied to other products, too.
Acknowledgements
This work was funded by the EU MACQU project contract No. AIR3-CT93-
1603. The funding is gratefully acknowledged.
References
AID, 1992. Quality standard for vegetables. Auswertungs- und Informationsdienst fu r Ernahrung,
Landwirtschaft und Forsten (AID) e.V. 53179 Bonn, Germany.
Choi, K., Lee, G., Han, Y.J., Bunn, J.M., 1995. Tomato maturity evaluation using colour image
analysis. Trans. Am. Soc. Agric. Eng. 38 (1), 171176.
Nielsen, H.M., Paul, W., 1995. Quality measurements of tomatoes by image analysis. Acta Hort. 421, 2.
Nielsen, H.M., Paul, W., 1996. Tomato quality: correlations between image analysis and destructive
tests. Paper AGENG 96, Madrid, 2326, September.
Sarkar, N., Wolfe, R.R., 1985a. Feature extraction techniques for sorting tomatoes by computer vision.
Trans. Am. Soc. Agric. Eng. 28 (3), 970979.
Sarkar, N., Wolfe, R.R., 1985b. Computer vision based system for quality separation of fresh market
tomatoes. Trans. Am. Soc. Agric. Eng. 28 (5), 17141718.
USDA, 1991. United States standards for grades of fresh tomatoes. USDA Agricultural Marketing
Service, Washington, DC.
.