LINE-SCAN SPECTRAL IMAGING SYSTEM FOR ONLINE POULTRY CARCASS INSPECTION*

jfpe_341 125..143

KUANGLIN CHAO1, CHUN-CHIEH YANG and MOON S. KIM USDA-ARS Food Safety Laboratory Henry A. Wallace Beltsville Agricultural Research Center Beltsville, MD 20705
Accepted for Publication August 27, 2008

ABSTRACT A line-scan spectral imaging system was developed for online identication of wholesome and unwholesome freshly slaughtered chicken carcasses on commercial processing lines. Hyperspectral images acquired by the imaging system for 5,549 wholesome and 93 unwholesome chicken carcasses on a 140 bird per minute (bpm) processing line were analyzed to optimize region of interest size and location and determine key wavebands by which to implement online multispectral inspection based on single-waveband intensity and a two-waveband ratio. Multispectral inspection algorithms were developed for real-time online chicken inspection. The imaging system inspected over 100,000 chickens on the processing line during continuous operation and accurately identied over 99% of wholesome and over 96% of unwholesome chickens. A system of this type can perform food safety inspection tasks accurately and consistently on high-speed processing lines (e.g., at least 140 bpm), to help poultry processors improve production efciency and satisfy increasing consumer demand for poultry products.

PRACTICAL APPLICATIONS The line-scan spectral imaging system was designed as a tool to assist poultry processors in meeting the requirements of the HACCP-Based Inspection Models Project as implemented by Food Safety and Inspection Service (FSIS). The system is particularly well suited for presorting poultry carcasses on high-speed processing lines by removing systemically diseased birds prior
* Mention of a product or specic equipment does not constitute a guarantee or warranty by the U.S. Department of Agriculture and does not imply its approval to the exclusion of other products that may also be suitable. 1 Corresponding author. TEL: 301-504-8450; FAX: 301-504-9466; EMAIL: kevin.chao@ ars.usda.gov Journal of Food Process Engineering 34 (2011) 125143. All Rights Reserved. Copyright the Authors Journal Compilation 2009 Wiley Periodicals, Inc. DOI: 10.1111/j.1745-4530.2008.00341.x

125

126

K.L. CHAO, C.-C. YANG and M.S. KIM

to the inspection stations. This can increase efciency and reduce crosscontamination risks by minimizing the presence and unnecessary processing of unwholesome birds on the processing line. Real-time data collection and prompt removal of unwholesome poultry carcasses can enhance the ability of FSIS to certify U.S. poultry products for both domestic and export markets. In addition, the spectral imaging methodology of this system has the potential for easy adaptation to other high-speed food processing tasks, particularly those involving automated inspection for quality indicators.

INTRODUCTION Today, the United States produces more poultry than any other country in the world, valued at more than $20 billion in on-farm revenues with broiler, egg, turkey and other poultry production combined (USDA 2007a). In recent years, broiler production has increased dramatically to meet rising market demand. Domestic per capita consumption of broilers increased from 59.5 pounds in 1990 to 76.9 pounds in 2000, and reached 87 pounds in 2006. U.S. poultry slaughter plants now process over 8.8 billion broilers annually (USDA 2007b). The 1957 Poultry Product Inspection Act mandated postmortem inspection of every bird carcass processed by a commercial facility. Since then, U.S. Department of Agriculture (USDA) inspectors have conducted on-site organoleptic inspection of all chickens processed at U.S. poultry plants for indications of diseases or defects. Inspectors of the USDA Food Safety and Inspection Service (FSIS) examine by sight and by touch the body, the inner body cavity surfaces, and the internal organs of every chicken carcass. With the 1996 nal rule on Pathogen Reduction and Hazard Analysis and Critical Control Point (HACCP) systems (USDA 1996), FSIS implemented the HACCP and Pathogen Reduction programs in meat and poultry processing plants throughout the country to prevent food safety hazards. More recently, FSIS has also been testing the HACCP-Based Inspection Models Project (HIMP) in a small number of volunteer plants (USDA 1997). HIMP requirements include zero tolerance for unwholesome chickens exhibiting symptoms of septox a condition of either septicemia or toxemia, which are systemic conditions characterized by pathogenic microorganisms or toxins in the bloodstream. USDA inspectors currently remove birds that exhibit signs of septox from the processing lines during bird-by-bird inspections conducted at a maximum speed of 35 birds per minute (bpm) for individual inspectors. Subject to human variability, the inspection process makes inspectors prone to fatigue and repetitive injuries, and the inspectors speed also limits the maximum possible output for the processing plants. This limit on production throughput, com-

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

127

bined with increasing chicken consumption and demand, places additional pressure on both chicken production and the food safety inspection system. During processing at a typical U.S. poultry plant, birds are rst slaughtered on kill lines and then transferred to evisceration lines on which inspection stations are located. Commercial evisceration lines in the U.S. currently may operate at speeds up to 140 bpm; however, such high-speed processing lines require up to four inspection stations, each with an FSIS inspector to conduct bird-by-bird inspection at the 35 bpm speed limit. Machine vision technologies, including red/green/blue color imaging, visible and near-infrared (Vis/NIR) spectroscopy and imaging, uorescence spectroscopy and imaging, and X-ray imaging, have been investigated for potential use in food and agricultural processing applications (Chen and Tao 2001; Delwiche 2003; Kim et al. 2003; Windham et al. 2003; Lu 2007). Spectral imaging techniques are particularly ideal for adaptation to online inspection applications (Lawrence et al. 2006; Park et al. 2006). Since 1998, researchers at the USDA Agricultural Research Service have been developing automated poultry inspection systems for high-speed online operations in the slaughter plant environment. Vis/NIR spectroscopy techniques were rst developed that could effectively differentiate wholesome and unwholesome chicken carcasses and viscera (Chao et al. 2003). Spectroscopy methods developed in the laboratory were adapted for use on processing lines. Spectral analysis methods were then modied for spectral image processing, and further advances in camera and computer equipment enabled implementation of automated imaging inspection on high-speed commercial processing lines. High-speed image-based inspection was tested successfully on a commercial 70 bpm evisceration line (Chao et al. 2007). With current evisceration lines operating at speeds as high as 140 bpm, there remains a need to develop imaging systems that can inspect chickens for wholesomeness at those high speeds. The objectives of this study were to implement a hyperspectral/multispectral line-scan imaging system for highspeed online operation in a commercial processing environment, to provide a method that can be used and modied for high-speed inspection tasks, and to evaluate the systems performance in differentiating wholesome and unwholesome chickens.

MATERIALS AND METHODS Operation of Spectral Imaging System The spectral imaging system (Fig. 1) consisted of an ElectronMultiplying Charge-Coupled-Device (EMCCD) camera (PhotonMAX 512b,

128

K.L. CHAO, C.-C. YANG and M.S. KIM

FIG. 1. A PHOTOGRAPH OF THE HYPERSPECTRAL/MULTISPECTRAL LINE-SCAN IMAGING INSPECTION SYSTEM ON A COMMERCIAL CHICKEN PROCESSING LINE

Roper Scientic, Inc., Trenton, NJ), an imaging spectrograph, a C-mount lens, and two pairs of high power, broad-spectrum white light-emitting-diode line lights. The EMCCD camera has approximately 512 512 pixels and is thermoelectrically cooled to approximately -70C (via a three-stage Peltier device). An imaging spectrograph (ImSpector V10OEM, Specim/Spectral Imaging Ltd., Oulu, Finland), and a C-mount lens (Rainbow CCTV S6x11, International Space Optics, S.A., Irvine, CA) are attached to the EMCCD imaging device. The spectrograph aperture slit of approximately 50 microns limits the instantaneous eld of view (IFOV) of the imaging system to a thin line for line-scan imaging. Light from the IFOV is dispersed by a prismgrating-prism line-scan spectrograph and projected onto the EMCCD imaging device. The spectrograph creates a two-dimensional (spatial and spectral) image for each line-scan, with the spatial dimension along the horizontal axis and the spectral dimension along the vertical axis of the EMCCD imaging device. Thus, for hyperspectral imaging, a full spectrum is acquired for every pixel in each line scan (Fig. 2). The spectral distribution of useful wavelengths and the size of the spatial image features to be processed determine the parameters for image binning, which reduces the number of image pixels and increases the signal-to-noise ratio by adding together photons from adjacent pixels in the detector array. For this study, the original hyperspectral line-scan image size (512 512 pixels) was reduced by 1 4 binning to result in line-scan images with a spectral resolution of 128 pixels (512 divided by 4) in

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

129

FIG. 2. HYPERSPECTRAL IMAGES ACQUIRED BY PERFORMING FULL-SPECTRUM DATA ACQUISITION FOR EVERY PIXEL IN EACH LINE-SCAN IMAGE

the spectral dimension. Then, because the useful spectrum of light did not span the entire width of the EMCCD detector, the rst 20 and last 53 spectral bands were discarded, resulting in a nal hyperspectral line-scan image size of 512 55 pixels. Hyperpspectral analysis of the images of whole chickens, compiled from hyperspectral line-scans acquired on a commercial chicken processing line, was performed to determine specic region of interest (ROI) parameters and wavebands useful for online wholesomeness inspection for chicken carcasses. The same line-scan imaging system could then be recongured in the settings for the camera control software to use only the selected wavebands for realtime multispectral inspection of chicken carcasses. Figure 3 shows examples of chicken images acquired by the imaging system in multispectral imaging mode on a chicken processing line. The imaging system is suitable for online multispectral inspection on high-speed chicken processing lines because of the capacity for short-exposure low-light imaging provided by the EMCCD detector. Pixels from the detector are binned by the high-speed shift register (which is built into the camera hardware) and transferred to the 16-bit digitizer, which has a rapid pixel-readout rate of approximately 10 MHz. The digitizer performs rapid analog-to-digital conversion of the image data for each line-scan image. The rapid multispectral image acquisition is followed by computer image analysis for real-time classication of wholesome and unwholesome pixels in the line-scan images of the chicken carcasses. Hyperspectral Image Analysis Hyperpsectral image analysis was performed using MATLAB software (MathWorks, Natick, MA). Image background removal was rst performed

130

K.L. CHAO, C.-C. YANG and M.S. KIM

400 nm (1th band)

580 nm (30th band)

620 nm (36th band) 750 nm (55th band)

Wholesome

Unwholesome

FIG. 3. CHICKEN CARCASS IMAGES AT FOUR SPECTRAL WAVEBANDS, ACQUIRED DURING MULTISPECTRAL LINE-SCAN IMAGING ON THE CHICKEN PROCESSING LINE

using a 0.1 relative reectance threshold value for the 620-nm waveband. For any pixel in a hyperspectral line-scan, if its reectance at 620 nm was below the threshold value, then the pixel was identied as background and its value at all wavebands was re-assigned to zero. The background-removed line-scan images were compiled to form images of complete chicken carcasses for both the wholesome bird and unwholesome bird data sets. These images were analyzed to determine the parameters for an optimized ROI for use in differentiating wholesome and unwholesome birds. Within a bird image, the potential ROI area spanned from an upper border across the breast of the bird to a lower border at the lowest nonbackground spatial pixel in each line scan, or to the last (512th) spatial pixel if there were no background pixels present at the lower edge of the image. For each potential ROI, the average relative reectance spectrum was calculated across all ROI pixels for all wholesome chicken images, and the average relative reectance spectrum was calculated across all ROI pixels for all unwholesome chicken images. The difference spectrum between the wholesome and unwholesome average spectra was calculated. This calculation was performed for all potential ROIs evaluated, which varied in size and were dened by the number of ROI pixels and their vertical coordinate locations within each line-scan. The optimized ROI was identied as being that which provided the greatest spectral difference between averaged wholesome pixels and averaged unwholesome pixels across all 55 wavebands. Using the optimized ROI, a key waveband was identied as being the waveband corresponding to the greatest spectral difference between averaged wholesome chicken pixels and averaged unwholesome chicken pixels, for differentiating wholesome and unwholesome chicken carcasses by relative reectance intensity. Again using the optimized ROI, the average wholesome

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

131

and average unwholesome spectra were examined for wavebands at which local maxima and minima occurred, to identify wavebands that might be used in two-waveband ratios for differentiating wholesome and unwholesome birds. The value of each potential band ratio was calculated for the average wholesome chicken pixels and for the average unwholesome chicken pixels. The two-waveband ratio showing the greatest difference in ratio value between average wholesome and average unwholesome chicken pixels was selected. Thus, multispectral imaging inspection used the selected key wavelength and the two-waveband ratio to differentiate between wholesome and unwholesome chicken carcasses. Multispectral Inspection of Chicken Carcasses The capacity to detect individual bird carcasses, classify the carcass condition, and generate a corresponding output useful for process control, at speeds compatible with online processing line operations, is required for effective multispectral imaging inspection for wholesomeness of chicken carcasses on a processing line. LabVIEW 8.0 (National Instruments Corp., Austin, TX) was used to control the spectral imaging system to perform these tasks. The following algorithm, outlined below and then discussed further, was developed to detect the entry of a bird carcass into the IFOV and was based on the imaging systems line-by-line mode of operation. Figure 4 shows the line-by-line algorithm for multispectral inspection to detect and classify wholesome and unwholesome chicken carcasses on a processing line. First, a line-scan image was acquired that contains only raw reectance values at the two key wavebands needed for intensity and ratio differentiation, the raw reectance data was converted into relative reectance data, and background pixels were removed from the image (Fig. 4, Box 4.1). The line-scan image was checked for the presence of the Starting Point (SP) of a new bird (Fig. 4, Box 4.2); if no SP was present, no further analysis was performed for this line-scan image and a new line-scan image was acquired. If the line-scan was found to contain an SP, then the ROI pixels were located (Fig. 4, Box 4.3) and the decision output value of Do was calculated for each pixel in the ROI of the line-scan image (Fig. 4, Box 4.4). With each new line-scan image acquired (Fig. 4, Box 4.5), the ROI pixels were located, and the decision output value of Do was calculated for each pixel, until the Ending Point (EP) of that bird was detected (Fig. 4, Box 4.6), indicating no additional line-scan images to be analyzed for that carcass. The average Do value for the bird was calculated (Fig. 4, Box 4.9) and compared to the threshold value (Fig. 4, Box 4.10) for the nal determination of wholesomeness or unwholesomeness for the bird carcass (Fig. 4, Boxes 4.11 and 4.12).

132

K.L. CHAO, C.-C. YANG and M.S. KIM

FIG. 4. A FLOWCHART OF THE METHOD FOR ONLINE MULTISPECTRAL LINE-SCAN IMAGING INSPECTION OF CHICKENS FOR WHOLESOMENESS

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

133

With the acquisition of each new line-scan at the start of the detection algorithm (Fig. 4, Box 4.1), the relative reectance at 620 nm was checked for each of the uppermost 256 pixels in the line-scan image. The relative reectance at 620 nm was always below 0.1 for these pixels when no chicken carcass was present in the IFOV. When the relative reectance at 620 nm increased above 0.1 for any single pixel among these 256 pixels in the linescan image, this indicated that a chicken carcass had entered the IFOV. The detection algorithm examined only the uppermost 256 pixels in order to avoid detecting carcass wings, which tended to overlap between adjacent birds on the processing line. After detecting a line-scan image with at least one pixel among the uppermost 256 exhibiting relative reectance greater than 0.1 at 620 nm, the subsequent line-scan images were monitored as additional pixels within the 256 pixels began showing relative reectance values greater than 0.1 (Fig. 4, Box 4.2). Between the rst detected pixel and the 256th pixel, pixels below the rst detected pixel would begin showing increased values at 620 nm as the chicken moved further across the eld of view. There would eventually be a line-scan image with only one (or several) remaining lowintensity pixel located below the rst detected pixel, and above or at the 256th pixel, that would be immediately followed by another line-scan in which the previous line-scan images last low-intensity pixel(s) had increased above 0.1. The last low-intensity pixel, or the pixel in the center of the last contiguous group of remaining low-intensity pixels, was identied as the SP of the bird carcass and represented the junction between the thigh and the abdomen on the leading edge of the carcass. Similar to the above algorithm, the following algorithm was developed to detect the last relevant line-scan image for each bird as it passed through the IFOV (Fig. 4, Box 4.6). After the SP was detected, each subsequent line-scan image was analyzed to determine if the relative reectance intensity at 620 nm for the pixel matching the vertical coordinate of the SP was above or below 0.1. When a line-scan image was acquired for which that pixel had relative reectance intensity at 620 nm that was below 0.1, this pixel was identied as the EP of the bird carcass, indicating that the main body of the bird had passed through the IFOV already and no further line-scans should be analyzed for that bird. Once the SP for a bird carcass was identied, the line-scan image containing the SP and subsequent line-scan images, up to the one containing the EP, were analyzed line-by-line (Fig. 4, Boxes 4.3 through 4.8), using the following algorithm to classify the condition of the bird. For each line-scan image, fuzzy logic membership functions were used to produce two decision outputs for each nonbackground pixel in the line-scan image that was located within the ROI, using the ROI and waveband parameters that were determined through hyperspectral image analysis. A fuzzy logic membership function was

134

K.L. CHAO, C.-C. YANG and M.S. KIM

FIG. 5. STRUCTURE OF THE FUZZY LOGIC MEMBERSHIP FUNCTIONS USED IN THIS STUDY TO OBTAIN TWO DECISION OUTPUTS FROM ONE INPUT VALUE I1: intensity-based input value; U1m: intensity mean for unwholesome chicken; U1sd: intensity standard deviation for unwholesome chicken; W1m: intensity mean for wholesome chicken; W1sd: intensity standard deviation for wholesome chicken; W1: fuzzy membership value for wholesome; U1: fuzzy membership value for unwholesome.

constructed using the mean and standard deviation values for relative reectance at the key wavelength for wholesome pixels (W1m and W1sd) and for unwholesome pixels (U1m and U1sd) within the optimized ROI, which was previously calculated during the hyperspectral image analysis (Fig. 5). For each pixel in the line-scan image, the relative reectance at the key wavelength was used as an input value, I1, to the fuzzy logic membership function to generate wholesome and unwholesome fuzzy membership values W1 and U1, respectively. Another fuzzy logic membership function was similarly constructed using the two-band ratio data; for each pixel in the line-scan image, the two-band ratio value was used as input value I2 to generate wholesome and unwholesome fuzzy membership values W2 and U2, respectively. The fuzzy inference engine executed a minmax operation (Chao et al. 1999) to obtain a decision output Do for each pixel based on the n membership functions as follows (in this case, n = 2):

Do = max (min [W1 . . . Wn ] min [U1 . . . Un ])

For each pixel, the value of Do occurred between 0 and 1, where 0 indicates 100% possibility of wholesomeness and 1 indicates 100% possibility of unwholesomeness. When the EP for that bird carcass was encountered, the average Do value for all ROI pixels for that bird was calculated (Fig. 4, Box 4.9). The bird was identied as being unwholesome if the average Do value

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

135

was greater than 0.6, a threshold value determined from previous 70 bpm test results of this system (Chao et al. 2007). Otherwise the bird was identied as being wholesome (Fig. 4, Boxes 4.10, 4.11, 4.12). In-plant Hyperspectral Imaging and Online Multispectral Inspection In-plant hyperspectral line-scan images of chickens were acquired on a 140 bpm commercial processing line in March 2007. A total of 5,549 wholesome and 93 unwholesome chickens were imaged, their conditions identied by an FSIS veterinary medical ofcer who observed the birds as they approached the illuminated IFOV of the imaging system. The 55-band hyperspectral data for the chicken carcasses were analyzed for ROI optimization and for selection of one key wavelength and two ratio wavebands based on average spectral differences between wholesome and unwholesome birds. Random track mode on the imaging system was implemented for multispectral inspection using only the key wavelength and ratio wavebands. LabView-based software modules were developed for detecting the SP and the EP of each bird, and for implementing classication algorithms based on fuzzy logic. The imaging system conducted online multispectral inspection for over 100,000 birds on a commercial processing line during two 8-hour shifts in July 2007. To verify system performance, an FSIS veterinary medical ofcer identied wholesome and unwholesome conditions of birds immediately before they entered the IFOV of the imaging system, during several 3040 min periods, for direct comparison with the classication results produced by the multispectral imaging system.

RESULTS AND DISCUSSION The hyperspectral images were analyzed to optimize ROI size and location and to select the wavebands for differentiation of wholesome and unwholesome birds by reectance intensity and by waveband ratio calculated from ROI pixels for each bird. A contour image of two example birds is shown in Fig. 6, with the SP and the EP marked on each. Within each line-scan, possible ROI pixels begin at the SPEP line and extend to the furthest nonbackground pixel below the SPEP line, which in some cases may coincide with the lowermost pixel of the line-scan image. Parameters m and n indicate the location of the upper and lower ROI borders for ROIs under consideration in this study, as percentages of the pixel length between the SPEP line and the furthest nonbackground pixel within each line-scan image. To optimize the ROI size and location, combinations of m and n, with values of m between 10% and 40% and values of n between 60% and 90%, were evaluated.

136

K.L. CHAO, C.-C. YANG and M.S. KIM

FIG. 6. CONTOUR IMAGES OF TWO CHICKEN CARCASSES MARKED WITH EXAMPLE LOCATIONS OF THE SP, EP, m AND n PARAMETERS USED FOR LOCATING THE REGION OF INTEREST

For each possible ROI, the average spectrum was calculated across all ROI pixels from the 5,549 wholesome chicken carcasses, and the average spectrum was calculated across all ROI pixels from the 93 unwholesome chicken carcasses. The difference between the average wholesome and average unwholesome value at each of the 55 bands was calculated. Figure 7 shows the range of these values for each possible ROI. Because the 4060% ROI showed the range with the greatest difference values between the average wholesome and average unwholesome spectra, this ROI was used in the waveband selection process. For this optimized ROI, consisting of the 4060% region of each line-scan image, Fig. 8 shows the average wholesome spectrum and average unwholesome spectrum as calculated across all line-scans in the data set. The 580-nm band showed the greatest difference between the average wholesome and the average unwholesome spectra, and thus, was selected as the key waveband to be used for intensity based differentiation of wholesome and unwholesome chicken carcasses. For differentiation of wholesome and unwholesome chicken carcasses by a two-waveband ratio, the six wavebands that were investigated are marked on Fig. 9, which shows the same average wholesome and average unwholesome spectra for the optimized ROI as Fig. 8. Because visual examination showed noticeable differences between the average wholesome and average unwhole-

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

137

FIG. 7. PLOT OF THE RANGE OF DIFFERENCE VALUES BETWEEN AVERAGE WHOLESOME AND AVERAGE UNWHOLESOME CHICKEN SPECTRA FOR ALL ROIS EVALUATED DURING HYPERSPECTRAL ANALYSIS TO OPTIMIZING THE ROI FOR MULTISPECTRAL INSPECTION OF CHICKENS

some spectral slopes in the three areas corresponding to 440460 nm, 500 540 nm, and 580620 nm, two-band ratios were investigated using these particular pairings. For these three two-waveband ratios (440 and 460 nm, 500 and 540 nm, and 580 and 620 nm), the average wholesome and average unwholesome ratio values were calculated. The differences in average ratio value between wholesome and unwholesome were then calculated:

W440 W460 U 440 U 460 = 0.003461 W500 W540 U 500 U 540 = 0.038602 W580 W620 U 580 U 620 = 0.115535
The last ratio, using the 580-nm and 620-nm wavebands, showed the greatest difference between the average wholesome and average unwholesome chicken spectra and was thus selected for use in differentiation by two-waveband ratio. The optimized ROI and key wavebands determined from the hyperspectral data analysis were used for online multispectral inspection of over 100,000

138

K.L. CHAO, C.-C. YANG and M.S. KIM

FIG. 8. THE AVERAGED WHOLESOME AND AVERAGE UNWHOLESOME CHICKEN SPECTRA OBTAINED DURING HYPERSPECTRAL ANALYSIS, HIGHLIGHTING THE 580-nm KEY WAVEBAND THAT CAN BE USED FOR INTENSITY-BASED DIFFERENTIATION OF WHOLESOME AND UNWHOLESOME CHICKENS

chickens on a 140 bpm processing line during two 8-hour shifts at a commercial poultry plant. Figure 10 shows examples of chicken images highlighting the ROI that was used for online inspection. The inspection program specically determined the 4060% ROI for each line-scan image as it was acquired, which was clearly affected by the size and position of the bird, and thus, could vary signicantly for different birds. For a bird whose body extended past the lower edge of the image, such as the rst bird in Fig. 10, the total ROI was a rectangular area. In contrast, irregularly shaped ROIs resulted for birds positioned such that background pixels were present at the lower edge of the image. Table 1 shows the mean and standard deviation values for the 580-nm key waveband and the 580-nm/620-nm two-waveband ratio from the hyperspectral analysis ROI data, which were used to build the fuzzy logic membership functions for online multispectral classication. Figure 5 shows the structure of the fuzzy logic membership function based on the 580-nm key wavelength, which was the same structure as that used for the two-waveband ratio fuzzy logic membership function. These functions were used to classify each ROI

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

139

FIG. 9. THE AVERAGED WHOLESOME AND AVERAGE UNWHOLESOME CHICKEN SPECTRA OBTAINED DURING HYPERSPECTRAL ANALYSIS, HIGHLIGHTING POSSIBLE KEY WAVEBANDS FOR TWO-WAVEBAND RATIO DIFFERENTIATION OF WHOLESOME AND UNWHOLESOME CHICKENS

FIG. 10. NINE CHICKEN IMAGES WITH THE OPTIMIZED ROI HIGHLIGHTED ON EACH CHICKEN

pixel within a line-scan image as either wholesome or unwholesome, by using each pixels 580-nm intensity value and its ratio value using 580 and 620 nm as inputs to obtain a decision output value Do between 0 and 1 for each criteria. The average of all the Do values calculated for all ROI pixels of a bird was used to determine a wholesome or unwholesome assignment by comparison with a threshold value of 0.6. The rst image in Fig. 11 (top) shows a masked image

140

K.L. CHAO, C.-C. YANG and M.S. KIM

TABLE 1. MEAN AND STANDARD DEVIATION VALUES FOR THE SINGLE-WAVEBAND REFLECTANCE INTENSITY AND FOR THE TWO-WAVEBAND RATIO OBTAINED DURING HYPERSPECTRAL IMAGE ANALYSIS OF WHOLESOME AND UNWHOLESOME CHICKENS Wholesome Mean l1 Rl1/l2 0.378 0.948 SD 0.088 0.037 Unwholesome Mean 0.243 0.904 SD 0.076 0.052

SD, standard deviation; l1, intensity at 580 nm; Rl1/l2, ratio of intensities at 580 nm and 620 nm.

FIG. 11. A MASKED IMAGE (TOP) OF NINE CHICKENS THAT HIGHLIGHTS THE ROI PIXELS TO BE ANALYZED FOR EACH CHICKEN, AND A SECOND IMAGE (BOTTOM) HIGHLIGHTING INDIVIDUAL PIXELS WITHIN THE ROI FOR EACH CHICKEN THAT WERE CLASSIFIED AS WHOLESOME

of nine chickens, highlighting all the ROI pixels for each bird. Two Do values were calculated for each of these ROI pixels, one for the key waveband and one for the two-waveband ratio. Online multispectral inspection averaged the Do values for all ROI pixels for each bird, in order to classify the bird by

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

141

TABLE 2. COUNTS OF WHOLESOME AND UNWHOLESOME BIRDS IDENTIFIED DURING INSPECTION SHIFTS BY PROCESSING LINE INSPECTORS AND BY THE IMAGING INSPECTION SYSTEM
Line inspectors Wholesome Shift 1 Shift 2 53,563 (99.84%) 64,972 (99.89%) Unwholesome 84 (0.16%) 71 (0.11%) Total 53,647 (100%) 65,043 (100%) Imaging inspection system Wholesome 45,305 (99.37%) 60,922 (99.84%) Unwholesome 288 (0.63%) 98 (0.16%) Total 45,593 (100%) 61,020 (100%)

comparison to the threshold value. For illustration purposes, the second image in Fig. 11 (bottom) highlights the classication results of classifying the individual pixels in the ROIs (instead of whole birds), obtained by averaging the two Do values for each ROI pixel in the top image and comparing the average value with the 0.6 threshold value. In this illustrative example, the fourth chicken from the left is an unwholesome bird and all of its ROI pixels were individually identied as unwholesome, consequently not appearing in the second image. Table 2 shows the total counts of chickens identied by the imaging system as being either wholesome or unwholesome during the two shifts of online multispectral inspection. Numbers drawn from FSIS tally sheets, created by three inspection stations on the same processing line during the same inspection shifts, are shown for comparison. Although direct bird-to-bird comparison between the imaging inspection system and the inspectors was not feasible, the percentages indicate that the relative numbers of wholesome and unwholesome birds identied by the imaging inspection system and by the processing line inspectors were not signicantly different. System verication was also performed for several 3040 min periods within the inspection shifts, by an FSIS veterinary medical ofcer. This consisted of bird-by-bird observation of chicken carcasses on the processing line immediately before they entered the IFOV of the imaging system. The imaging system output was observed for comparison with the veterinary medical ofcers identications. Over four verication periods during inspection shift 1, the imaging system correctly identied 16,056 of 16,174 wholesome birds (99.27% correct) and 41 of 43 unwholesome birds (95.35% correct). Over six verication periods during inspection shift 2, the imaging system correctly identied 27,580 of 27,626 wholesome birds (99.83% correct) and 34 of 35 unwholesome birds (97.14% correct). These results, together with the percentages listed in Table 2, strongly suggest that the imaging inspection system can perform successfully on a high-speed commercial poultry processing line. For multispectral inspection conducted on the 140 bpm processing line in this study, the imaging system acquired about 3040 line-scan images between

142

K.L. CHAO, C.-C. YANG and M.S. KIM

the SP and EP for each chicken inspected. Previous testing of the imaging system (Chao et al. 2007) demonstrated similar performance in identifying of wholesome and unwholesome birds on a 70 bpm processing line, with the multispectral classication based on about 7080 line-scan images for each chicken. Because this type of unwholesome bird exhibits a systemic unwholesome condition affecting the entire body of the bird, this line-scan imaging system is capable of identifying such birds at even higher speeds. For example, on a 200 bpm processing line, the system would perform similarly in identifying wholesome and unwholesome birds by analyzing about 2025 line-scan images for each chicken. CONCLUSIONS An online line-scan imaging system capable of both hyperspectral and multispectral Vis/NIR imaging was developed to inspect freshly slaughtered chickens on a high-speed processing line for wholesomeness. During continuous in-plant operation in multispectral imaging inspection mode, the system inspected over 100,000 chickens on a 140 bpm processing line and accurately identied over 99% of wholesome chickens and over 96% of unwholesome chickens. These results demonstrated that with appropriate hyperspectral analysis methods and multispectral inspection algorithms for online operation, a line-scan spectral imaging system utilizing an EMCCD camera can perform food safety inspection tasks accurately and consistently while meeting the high-speed production requirements (e.g., at least 140 bpm) of commercial chicken processing. The imaging inspection system can be used as a tool to help poultry processors improve efciency and lower the risks of crosscontamination on the processing line, by removing most unwholesome chicken carcasses earlier on the processing line such that they are not presented for inspection and further unnecessary processing. REFERENCES CHAO, K., CHEN, Y.R., EARLY, H.L. and PARK, B. 1999. Color image classication systems for poultry viscera inspection. Appl. Eng. Agric. 15(4), 363369. CHAO, K., CHEN, Y.R. and CHAN, D.E. 2003. Analysis of Vis/NIR spectral variations of wholesome, septicemia, and cadaver chicken samples. Appl. Eng. Agric. 19(4), 453458. CHAO, K., YANG, C.C., CHEN, Y.R., KIM, M.S. and CHAN, D.E. 2007. Hyperspectral-multispectral line-scan imaging system for automated poultry carcass inspection applications for food safety. Poult. Sci. 86(11), 24502460.

AUTOMATED POULTRY CARCASS INSPECTION SYSTEM

143

CHEN, Z. and TAO, Y. 2001. Multi-resolution local multi-scale contrast enhancement of X-ray images for poultry meat inspection. Appl. Opt. 40(8), 11952000. DELWICHE, S.R. 2003. Classication of scab- and other mold-damaged wheat kernels by near-infrared reectance spectroscopy. Trans. ASAE. 46(3), 731738. KIM, M.S., LEFCOURT, A.M. and CHEN, Y.R. 2003. Multispectral laserinduced uorescence imaging system for large biological samples. Appl. Opt. 42(19), 39272934. LAWRENCE, K.C., WINDHAM, W.R., SMITH, D.P., PARK, B. and FELDNER, P.W. 2006. Effect of broiler carcass washing on fecal contaminant imaging. Trans. ASABE. 49(1), 133140. LU, R. 2007. Nondestructive measurement of rmness and soluble solids content for apple fruit using hyperspectral scattering images. Sens. Instrumen, Food Qual. 1(1), 1927. PARK, B., LAWRENCE, K.C., WINDHAM, W.R. and SMITH, D.P. 2006. Performance of hyperspectral imaging system for poultry surface contaminant detection. J. Food Eng. 75(3), 340348. USDA. 1996. Pathogen reduction: Hazard analysis and critical control point (HACCP) systems. Final rule. Fed. Regist. 61, 3880538989. USDA. 1997. HACCP-based inspection models project (HIMP). Proposed rule. Fed. Regist. 62, 3155331562. USDA. 2007a. Livestock, Dairy, & Poultry Outlook. Economic Research Service, Washington, DC. USDA. 2007b. Poultry Production and Value 2006 Summary. National Agricultural Statistics Service, Washington, DC. WINDHAM, W.R., SMITH, D.P., PARK, B., LAWRENCE, K.C. and FELDNER, P.W. 2003. Algorithm development with visible/nearinfrared spectra for detection of poultry feces and ingesta. Trans. ASAE. 46(6), 17331738.