You are on page 1of 6

Aquacultural Engineering 67 (2015) 8–13

Contents lists available at ScienceDirect

Aquacultural Engineering
journal homepage: www.elsevier.com/locate/aqua-online

An automatic counting system for transparent pelagic


fish eggs based on computer vision
Yane Duan a,b , Lars Helge Stien c , Anders Thorsen c , Ørjan Karlsen c , Nina Sandlund c ,
Daoliang Li a,∗ , Zetian Fu a,∗ , Sonnich Meier c
a
College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, PR China
b
College of Computer and Information Engineering, Beijing Agricultural University, Beijing 102206, PR China
c
Institute of Marine Research, Austevoll Research Station, 5392 Storebø, Norway

a r t i c l e i n f o a b s t r a c t

Article history: Alive eggs of marine species with pelagic eggs float, while dead eggs usually sink. A non-invasive method
Received 8 January 2015 for counting the number of floating eggs therefore gives the possibility to track survival throughout
Accepted 5 May 2015 experiments. In this paper we present an automatic image analysis method for counting live pelagic eggs
Available online 12 May 2015
of marine fish. Pelagic fish eggs are typically transparent and difficult to detect in images. Current image
analysis methods for counting pelagic fish eggs are therefore done on eggs transfixed in a polymer to
Keywords:
create contrast between the eggs and the background. This kills the eggs. The main advantage of the
Pelagic fish eggs
presented method is that it is non-invasive and only requires a minimum of handling of the eggs. As case
Automatic counting system
Computer vision
studies we collected images of Atlantic haddock (Melanogrammus aeglefinus) and Atlantic cod (Gadus
Watershed method morhua) eggs. The eggs in the images were manually counted for verification of the methodology. The
Image processing average counting error of false positives was 6% and the average counting error of false negatives was
2%. This demonstrates that the method is objective and accurate.
© 2015 Published by Elsevier B.V.

1. Introduction from total sample weight or volume is far less laborious, but also
less accurate and in addition requires a measurement of dry egg
Pelagic fish eggs are typically transparent and float near the sur- volume and this is rough treatment of living fish eggs (Browne,
face, while eggs that die typically sink (Pelster, 1997). This entails 2009). In addition, some automated electronic methods and devices
that counting the number of floating eggs may be used to track for counting fish egg have been described (Witthames and Walker,
survival rate of eggs throughout experiments. Traditionally, the 1987). These, however, rely on object singulation and size unifor-
quantity of fish eggs is determined either by counting them one by mity which may limit their value for commercial operations (Zion,
one using a pipette, or by estimating the number of eggs by mea- 2012).
suring the total volume or weight and calculate the numbers from An alternative to the traditional methods of counting eggs is
established correlations between volume/weight and egg num- digital image analysis. Pohar and Pen (1983) used photocopying
bers (Simpson, 1951; Davies and Paulik, 1965; Boyar and Clifford, for counting salmonid eggs. Thorsen and Kjesbu (2001) produced
1967; Dodgshun, 1980; Parrish et al., 1980; Tanasichuk et al., images of oocytes fixed in formaldehyde by illuminating them from
1985; Witthames and Walker, 1987; Kjesbu, 1989; Musonda, 1999; behind. These oocytes were in the vitellogenic stage where they
Gundersen et al., 1999; Hinshaw and Thompson, 2000). Counting have a brownish color and therefore easily stand out from a bright
using a pipette produces an exact count of the eggs, but requires background in contrast to fertilized eggs which are transparent.
an experienced operator, that the eggs are dissociated and that the Friedland et al. (2005) developed an image analysis method for
sizes of eggs are just below the diameter of the tube. Counting this scanned images of fish eggs in Petri dishes. This method first trans-
way is tedious and time-consuming. Estimating the number of eggs fixes the eggs in a polymer before they are scanned. The method
requires that the eggs are loosely distributed and not too close to
the sides of the Petri dish. The back part of the Petri dish is painted
∗ Corresponding authors at: P.O. Box 121, China Agricultural University,
black and the result of the treatment is an image where the eggs
stand out as white discs in front of a black background. The eggs can
17 Tsinghua East Road, Beijing 100083, PR China. Tel.: +86 10 62736764;
fax: +86 10 62737741. therefore relatively easily be identified by automatic image analy-
E-mail addresses: dliangl@cau.edu.cn (D. Li), fzt@cau.edu.cn (Z. Fu). sis algorithms, and hence counted. Mello et al. (2009) developed a

http://dx.doi.org/10.1016/j.aquaeng.2015.05.001
0144-8609/© 2015 Published by Elsevier B.V.
Y. Duan et al. / Aquacultural Engineering 67 (2015) 8–13 9

Fig. 1. Schematic outline of the main hardware components of the system. The
camera is mounted 160 mm above the Petri dish and the light tube illuminates the
Petri dish 130 mm from the side.

semi-automatic method to count the number of eggs in ovitraps


images using image processing, particularly color segmentation
and mathematical morphology-based non-linear filters.
The above-mentioned image analysis methods use fixated
material and can therefore not be applied on living eggs. The main
objective of this paper is to develop a non-invasive automatic
method to count living eggs by using computer vision technology.

2. Materials and methods

2.1. Eggs and sampling

Haddock eggs were collected on 28 March 2014 from spawn-


ing haddock at the Institute of Marine Research’s research station
in Austevoll, Norway. The eggs were carefully retrieved from egg
collection tanks using a 10 ml plastic cylinder with a plankton-net
in the bottom (180 ␮m). The samples were first quickly drained for
water and the volume (precision 0.1 ml) measured before the eggs
were poured into a Petri dish with some seawater (2–5 ml eggs
in each Petri dish, a total of 113 samples). The Petri dishes were
photographed, and the eggs immediately transferred to experimen-
tal tanks for continued incubation. Egg numbers were also counted
manually on 32 photographs. In addition, to test the image analy-
sis method (presented below) with eggs from another marine fish
with transparent pelagic eggs, newly fertilized eggs from Atlantic
cod were poured into Petri dish with some seawater and photo-
graphed (1–5 ml eggs in each Petri dish, a total of 15 samples).
The number of eggs counted by means of the image analysis were
compared with egg numbers estimated from measured volumes
using the calibration from Kjesbu (1989): number of egg per ml
(n) = 1222 × D−2.71 . The average egg diameters (D) of the haddock
egg were 1.43 ± 0.06 mm (estimated egg number per ml = 464)
and for the cod eggs 1.10 ± 0.02 mm (estimated egg number per
ml = 943).

Fig. 2. Original images, result after image pre-processing and result after segmen-
2.2. Automatic counting system tation of the eggs from the background. (A) Original image of Petri dish with sparse
distribution of fish eggs. (B) Original image of Petri dish with dense distribution of
2.2.1. Hardware of system fish eggs. (C, D) Gray scale versions of the images in (A) and (B), but where all pixels
The hardware of the automatic counting system for transpar- outside the ROI (indicated by the green circle) have been set to black. (E, F) Results
after morphological operations and gamma correction to increase contrast between
ent fish eggs includes an adjustable bracket, a color digital camera
the eggs and the background and remove noise. (G, H) Results after binarization and
(Canon EOS 650D, Cannon Technologies Corp., Japan), a light tube morphological operations. (I, J) Results after watershed operation. (K, L) Illustrations
(38 W, ∼50 Hz, l = 290 mm), a Petri dish (∅ = 136 mm, h = 20 mm), of final result.
and a personal computer (the core of the system is a ThinkPad X230
(Lenovo, China), 2 GB RAM, 500 GB HD, with a window 7 Operating
System) (Fig. 1).
The camera is mounted 160 mm directly over the Petri dish, fac-
ing downwards. To make the transparent eggs more visible, there
10 Y. Duan et al. / Aquacultural Engineering 67 (2015) 8–13

is a black paper beneath the Petri dish and the light tube is placed 2.2.4. Segmentation of the eggs from the background
130 mm horizontally to the side of the Petri dish lighting the eggs. For improving the accuracy of identification and segmentation
of the eggs from the background, the pre-processed images were
automatic classified into two classes according to their density:
2.2.2. Image acquisition
class 1 images with low density of eggs (less than 0.5 ml), Fig. 2A,
Each sample of fish eggs is poured into the Petri dish with 15 mm
and class 2 images with high density of eggs (more than 0.5 ml),
depth of seawater and then carefully dispersed in the Petri dish
Fig. 2B. For these two kinds of images, the comprehensive appli-
to be as loosely distributed as possible and to avoid eggs being
cation of multi-methods (thresholding, morphological operations
occluded by others in the image. A little experimentation with the
and watershed segmentation) was used. The process diagram is
haddock eggs showed that it was possible to achieve a good dis-
shown in Fig. 4. In the following we will describe the key operations
persion in the Petri dish for up to 2500 eggs (∼5 ml), and setting
in this flowchart.
the camera to an aperture of 5.6 mm, ISO 400 and an exposure time
of 1/30 s provided satisfactory images of the eggs (Fig. 2A and B).
2.2.5. Image binarizing by auto thresholding algorithm
During image acquisition, a total of four Petri dishes of same size
Due to several uncontrolled reasons (different location and
have been used repeatedly, and 113 images were captured.
numbers of eggs, previous and unequal light exposure, etc.) the
image mean gray level intensity varied between the images, also
2.2.3. Image pre-processing after pre-processing. To correct for this, an adaptive thresholding
All the referred functions are from MATLAB R2012b including (Otsu, 1979) was used to binarize the images (Fig. 2E and F) into
the Image Processing Toolbox (The MathWorks Inc., Natick, MA, pixels representing eggs and pixels representing background:
USA). The values of the parameters were determined by trial. Since 
the position of the Petri dish is the same from image to image, a 1 ≥ T = graytresh (f6 )
g1 = (1)
region of interest (ROI) can be automatically defined inside the 0 otherwise
edges of the Petri dish (Fig. 2C and D). This ensures that any specular
reflection on the sides of the Petri dish or from water droplets on the where g1 is the resulting binary image, T is the threshold generated
black base outside the dish do no influence the result. Gray mor- by the graythresh MATLAB-function and f6 is the output image from
phological operations and enhance processing is then performed the image preprocessing operations.
on a gray scale representation of the ROI. The process diagram is
shown in Fig. 3. 2.2.6. Morphological operations of bi-level image
For gray morphological operations, first the f1 images is After initial binarization by thresholding some fish egg objects
bottom-hat transformed f2 = imbothat (f1 , 10) to produce a frame may have holes, some small noise may still exist, part of the fish
representing the change in illumination across the scene, then f2 is eggs may be cut out, some eggs may be touching, etc. In order to
used to perform flat field correction: f3 = f1 − f2 . At the same time correct these, a fill holes operation was executed: h1 = imfill (g1 ,
an opening operation is executed on the f1 image: f4 = imopen (f1 , ‘holes’). Followed by an area opening operation h2 = bwareaopen
strel (‘disk’, 5)). Finally, to make the eggs stand out uniformly from (h1 , size) to remove small objects from the binary image, the size
the background, an additional operation was executed between f3 filter was defined by trial. All objects less than this size were con-
and f4 :f5 = f3 + f4 . The last step of the image preprocessing was to sidered as noise and removed to background. In our experiments,
use gamma correction to make the eggs stand out even more and this size was set to 80 pixels. Any connected region with pixels area
at the same time suppress noise (Fig. 2E and F): f6 = imadjust (f5 , ), <80 was deleted from the binary image h1 . For images of class 1,
where  is set to 0.5 (Gonzalez and Woods, 2008; Gonzalez et al., dilatation was then executed to ‘grow’ and ‘thicken’ objects so that
2009). divided parts of eggs were connected. Thereafter fill holes and small
objects removal procedures were performed again. Lastly, for both
image classes, an opening operation was used to remove, break and
diminish false connections between egg objects.
Orignal RGB Image
For setting of the structural element in the opening and dilata-
tion operations, we assumed that the fish eggs have a regular shape,
Converting RGB to Gray Image between elliptical and circular. To avoid the loss of small eggs in the
class 1 images, the structural disk element size was set to 6 × 6
Creating Image of Interest Area pixels for the dilation operation, and 7 × 7 pixels for the open-
ing operation. For class 2 images, the structural element size of
the opening operation was set to 6 × 6 pixels. These settings were
Gray Morphological Operations
established by trial and error (Fig. 2G and H).
Bottom-Hat Opening
Transformation Operation 2.2.7. Watershed segmentation
After the morphological operations, the binary images may still
Subtraction Operation have eggs that are connected. To avoid over-segmentation, the pre-
to Reduce the Noise sented method first creates a binary image b1 containing only egg
regions smaller or equal to the typical egg size and a binary image
Addition Operation to Enhance b2 containing only egg regions larger than the typical egg size. By
Egg Signal trial, based on the typical egg sizes, the size for images of class 1
was set to 1700 pixels, and for images of class 2 it was set to 800
pixels. The method then uses watershed segmentation to divide the
Gamma Enhance
regions in b2 :b2 ’ = watershed (bwdist (b2 )), and finally combines b1
and b2 ’ to create the final binary image b3 = b1 + b2 ’ (Fig. 2I and J). For
Storage of Preprocessing Result Image example, for Fig. 2G, the created images: b1 , b2 and b2 ’ are shown
in Fig. 5. The counting result of Fig. 2G is 93, and the result after
Fig. 3. Flowchart of image pre-processing. watershed segmentation (Fig. 2I) is 123.
Y. Duan et al. / Aquacultural Engineering 67 (2015) 8–13 11

Fig. 4. Flowchart of image identification and segmentation.

Fig. 5. Result images of dividing Fig. 2G, and result after watershed operation.

2.2.8. Counting by connected components that the eggs got a clearly distinct purple color (Fig. 2K and L).
Finally, at this stage the number of identified eggs is obtained With this image we automatically identified the purple eggs and
by utilizing a labeling operation (bwlabel, MATLAB R2012b). How- manually counted the residual eggs using the open source image
ever, in order to be able to do manual correction, the processed analysis program ImageJ v. 1.48 (National Institutes of Health,
binary image was instead combined with its original image, so USA).
12 Y. Duan et al. / Aquacultural Engineering 67 (2015) 8–13

3. Result and discussion

Because the research object is living eggs, one cannot do any-


thing to rectify the transparent nature of the eggs; they must also
remain in saltwater at all times and be handled as little and gently
as possible. This makes the image analysis more complicated.

3.1. Manual vs automatic counts

For calibration and verification of algorithms, we correlated the


numbers estimated using the image analysis with manual counted
eggs in 32 samples. The correlation showed that the automatic
counts were accurate, showed by high correlation (R2 = 0.991)
between the two methods (Fig. 6).
It should be noted that the overall error rate was not large: the
maximum counting error was 10%; the average counting error 2%; Fig. 6. Comparison between manually counted haddock egg numbers and numbers
counted using the image analysis (n = 32).
the average counting error of false positives 2%; and the average
counting error of false negatives 6%. Errors in object detection and
segmentation were a major cause of error in the computer count. volume measurement in this experiment were done very quickly in
Typically the false positives were either caused by a low contrast order to minimize the time the egg were out of the water (<0.5 min),
between fish egg and the background, or due to serious adhesions and did not use any force to remove water, only gravity. One source
of fish eggs. Errors of false negatives were often caused by the to uncertainty in volume measurement may therefore be varia-
existence of strong noise clusters, or existing over-segmentation. tion in the packing of the eggs in the cylinder, and this can also
Haddock eggs are sticky, and hence adhesions of fish eggs will occur explain the overestimation of the egg number from the volume.
and this will reduce the accuracy of the algorithm. The calibration formula of Kjesbu (1989) assumes that the volume
measurement are done close to the optimum packing the egg.
3.2. Counts vs estimates based on volume
3.3. Experimental analysis and discussion
The automatic egg count was also compared with egg number
estimated using volume and a calibration formula (Kjesbu, 1989). For the automatic counting of eggs the processing time will vary
The correlation between the numbers estimated from volumes and with the speed of the computer. With our laptop each image took
the automatic counting was high (Fig. 7). However, the estimates about 30 s to process. This processing did take place unattended by
based on volume overestimated egg numbers both for the large the operator and was therefore not man-hour consuming. For the
haddock eggs (114 ± 7%) and the smaller cod eggs (117 ± 9%). The manual counting, the man-hour consumption was very large, when

Fig. 7. Comparison between automatic counted eggs and number of egg estimated from the egg volume. A1 shows counted egg number relative to volume of haddock eggs
(n = 113) and A2 shows corresponding plot of counted egg number estimated from egg volume. B1 and B2 show similar plots for cod (n = 15).
Y. Duan et al. / Aquacultural Engineering 67 (2015) 8–13 13

eggs number is about 100, it need about 2 min to count. If more than Ove Utskot for management of the broodstock fish, and Michal
100 eggs, it will become very difficult to count manually. Also, the Rejmer and Maksan Domagoj for assistance in the laboratory work.
results indicated that Atlantic cod eggs, which are also transparent
and pelagic, can be counted with similar speed and accuracy as References
haddock eggs.
The results of our experiments demonstrate that our algorithms Boyar, H.C., Clifford, R.A., 1967. An automatic device for counting dry fish eggs. Trans.
Am. Fish. Soc. 96, 361–363.
are reliable, flexible, non-destructive and rapid compared with Browne, R., 2009. Counting Eggs and Larvae. Royal Zoological Society of Antwerp,
manual methods. However, our experiment also demonstrates that Antwerp, Belgium http://aark.portal.isis.org/researchguide/methods
the system has some limitations. Firstly, in the process of removing Davies, A.S., Paulik, G.J., 1965. The design operation and testing of a fish egg counter.
Prog. Fish-Cult. 21, 185–192.
specular reflections occurring at the edge of the Petri dish, some Dodgshun, T.J., 1980. Simple, accurate method of counting fish eggs. Prog. Fish-Cult.
eggs located in the reflections will be hard to detect. Secondly, the 42, 237–238.
contrast of the original RGB is a little low because of the high trans- Friedland, K.D., Ama-Abasi, D., Manning, M., Clarke, L., Kligys, G., Chambers, R.C.,
2005. Automated egg counting and sizing from scanned images: rapid sam-
parency of fish eggs. These factors all increased the difficulty of
ple processing and large data volumes for fecundity estimates. J. Sea Res. 54,
image processing and also affected the accuracy of identification 307–316.
and counting. To improve the method we will in the future try to Gonzalez, R.C., Woods, R.E., 2008. Digital Image Processing, 3rd ed. Prentice Hall,
optimize image acquirement by using a digital camera of higher New Jersey, USA, pp. 976.
Gonzalez, R.C., Woods, R.E., Eddins, S.L., 2009. Digital Image Processing Using MAT-
resolution, better sample illumination, and finding a good method LAB, 2nd ed. Prentice Hall, Upper Saddle River, NJ, pp. 827.
to avoid eggs close to the edge of dish. Gundersen, A.C., Kjesbu, O.S., Nedreaas, K.H., Stene, A., 1999. Fecundity of Northeast
Arctic Greenland halibut (Reinhardtius hippoglossoides). J. Northw. Atl. Fish. Sci.
25, 29–36.
4. Conclusion Hinshaw, J.M., Thompson, S.L., 2000. Trout Production Handling Eggs and Fry. North
Carolina State University, Southern Regional Aquaculture Centre Publications,
Based on Otsu’s auto threshold method, morphological opera- No. 220.
Kjesbu, O.S., 1989. The spawning activity of cod, Gadus morhua L. J. Fish Biol. 34,
tions, and watershed algorithm, we have developed an automatic 195–206.
non-invasive counting method for transparent pelagic fish eggs, Mello, C.A.B., Dos Santos, W.P., Rodriguez, M.A.B., Candeias, A.L.B., Gusmão, C.M.G.,
which therefore permit continued incubation after counting. The Portela, N.M., 660 pp 2009. Automatic counting of Aedes aegypti eggs in images
of ovitraps. In: Recent Advances in Biomedical Engineering. InTech, ISBN 978-
method was developed using Atlantic haddock eggs, but the 953-307-004-9.
method was also tested successfully on cod eggs. We believe the Musonda, F.F., 1999. Relationship of fecundity and body size of Lates stappersii in
method may be applied to many other transparent pelagic fish central lake Tanganyika, East Africa. Rep. Dept. Fish. Zambia, 1–4.
Otsu, N., 1979. A threshold selection method from gray-level histograms. IEEE Trans.
eggs as well. To verify the accuracy of the method, a comparison Syst. Man. Cybern. 9, 62–66.
between the automatic and manual counting was performed. The Parrish, B.B., Baxter, I.G., Mowat, M.J.D., 1980. An automatic fish egg counter. Nat.
average counting error of the automatic algorithms was 2%, and this Lond. 185, 777.
Pelster, B., 1997. Buoyancy at depth. In: Randall, D., Farrell, A.P. (Eds.), Deep-Sea fish.
is for many types of works well within acceptable levels. Compared
Academic Press, San Diego, pp. 195–237.
with traditional manual counting methods, the automatic count- Pohar, J., Pen, A., 1983. Use of photocopying for counting salmonid eggs. Prog. Fish-
ing system had the main advantage of being much less man-hour Cult. 45, 234–236.
consuming. Simpson, A.C., 1951. The spawning of the plaice (Pleuronectes platessa) in the North
Sea. Fish. Invest. Lond. Ser. 2 (22), 1–111.
Tanasichuk, R.W., Armstrong, C., Ware, D.W., 1985. An improved photo-electric fish
Acknowledgements egg counter. Can. J. Fish. Aquat. Sci. 42, 1255–1258.
Thorsen, A., Kjesbu, O.S., 2001. A rapid method for estimation of oocyte size and
potential fecundity in Atlantic cod using a computer-aided particle analysis
This work was supported by the Norwegian Research Council system. J. Sea Res. 46, 295–308.
(Project no. 234367), the Norwegian Academy of Science and Let- Witthames, P.R., Walker, M.G., 1987. An automated method for counting and sizing
ters, by the VISTA Foundation (Project no. 6161) and the Intelligent fish eggs. J. Fish Biol. 30, 225–235.
Zion, B., 2012. The use of computer vision technologies in aquaculture – a review.
Sensor Network and System Technologies for Fish Farming (Project Comput. Elect. Agric. 88, 125–132.
no. PIRSES-GA-2011-293422). We would like to acknowledge Stig

You might also like