Professional Documents
Culture Documents
Cristiano C. de Araujo
Abel G. S. Filho
Federal University
of Pernambuco - UFPE
Recife, Pernambuco, Brazil
Email: cca2@cin.ufpe.br
Federal University
of Pernambuco - UFPE
Recife, Pernambuco, Brazil
Email: agsf@cin.ufpe.br
I. I NTRODUCTION
Representation
Segmentation
Preprocessing
Image Acquisition
Level Processing
Description
High
Recognition
Intermediate
Interpretation
Knowledge Base
Automated Visual Inspection (AVI) for dimensional inspection is one of the main machine vision applications. AVI
enables inspections in hazardous or hard-to-access environments and can improve productivity and quality in industrial
production lines [1]. For instance, the AVI system implemented by C. Fernandez [2] made orange classification 27
times faster and mistakes reduced by 65% when compared to
human inspection.
Despite the relevance of AVI for industrial automation, it
still faces some challenges. Many AVI systems are based on
computationally intensive algorithms and expensive hardware.
This limits the application of AVI to certain domains. Thus
there is a growing demand for simpler algorithms implemented
in efficient embedded systems.
In order to improve the inspection rate and the image
resolution, this paper presents an implementation in FPGA
of the LEDges technique. We applied it as an AVI system to a
real industrial problem: flaw detection on edges of toothpaste
tubes. We measured and compared some parameters of the
developed AVI system with other related works. Compared
to the ARM-based implementation of the LEDges [3], the
results were much better. But these comparisons are not trivial
when other techniques are used since every inspection has
its specific requirements and we cannot test these approaches
using an identical test set. However, these measures show that
the developed system has a very good inspeciton rate.
This paper is structured as follows. We explain the image
processing pipeline in section II while in section III we
describe the LEDges technique. We show the architecture of
Results
Low
Problem Domain
Fig. 1.
Finally, the knowledge base contains accumulated knowledge about the inspection, provided by man and / or by
machine [4]. For instance, accumulated experiences of staff
in relation to an manual inspection to be automated, which
are already known results related to the problems and their
solutions.
III. LED GES
The LEDges [3] is a AVI technique for real time automated
visual dimensional inspections which is based on simplicity
of the algorithm and low complexity of hardware presented
in [5][10], but with the improvement of detecting all edges
of the object in the images acquired by the camera, not
just the outermost edges. Previously, this capability was only
possible in the works based on depth cameras, laser and projectors [11][14], which has a long processing time, resourceintensive computing, and delicate and complex image capture
devices. This makes it unfeasible their use in embedded and
real-time AVI systems with the following requirements: (i)
Simple classification of the object between good or flaw; (ii)
Fast response time without the use of high speed cameras;
(iii) A fuzzy background with many objects making the image
processing harder; (iv) A low space available to assemble the
new AVI system to preexisting production machinery. To the
best of our knowledge there are no solutions available that
meet all these requirements.
The LEDges is described in four steps as presented in
Figure 2. First the object is illuminated with a high power
structured light source and its image is captured by the camera.
The result of this step is an image displaying the lighted areas
of the objects in the foreground in higher intensities, while the
background and the shadows are displayed in lower intensities.
In the thresholding step the acquired image is easily segmented into two levels of intensity: (i) the background and
the shadows are indicated by regions with zero intensity, color
black, (ii) and light areas in the foreplane are indicated by the
maximum intensity, color white.
In the third step, signature generation, the thresholded image
is represented by its edges and described as a signature.
Each edge is represented numerically in terms of the distance
between the beginning of the image lines and relevant blackto-white and white-to-black transitions of the thresholded
image. This set of distances makes the image signature. Note
that, in general, the Hough transform, used in [15], [16] would
require greater use of hardware.
The last step is the comparison, which compares the generated signature with a standard good objects signature and
Scene
Input
Image
capture
Thresholding
Signature
generation
Processing Steps
Fig. 2.
Comparison
Result
Output
Fig. 3. Computer modeling of the LEDges on FREEMAT software. The acquired and thresholded image are displayed, as well as the image description as
a signature.
architecture based on the low cost FPGA TERASIC kit DE270, the LEDges can be applied to process images with good
resolution (640 x 480 pixels), detecting dimensional errors of
0.5 mm for the mechanical assembly performed and with a
rate of 100 inspections per second. Finally, computer modeling
allows a fast implementation in FPGA with good flexibility
to update the parameters. Therefore, the choice of an FPGAbased architecture achieves the objectives for the design.
V. FPGA- BASED ARCHITECTURE
The available hardware is the DE2-70 development kit,
including the D5M digital camera and LTM LCD display,
all from the Taiwanese company TERASIC. The DE2-70 is
equipped with a low cost FPGA Cyclone II EP2C70F896C6.
In addition to the components already available in the kit, it
was also necessary to install a high-power LED, required to
perform the LEDges illumination [3].
The AVI system was segmented into six major functional
blocks: (i) image acquisition; (ii) thresholding; (iii) signature
generation; (iv) comparison; (v) LCD displaying. Each block
comprises one or more hardware modules implemented in
FPGA as presented in Figure 4. The figure also shows the
inputs and outputs of the modules.
VI. C ASE STUDY: FLAW DETECTION ON EDGES OF
TOOTHPASTE TUBES
not adjusted, (i) image resolution and (ii) camera focus, once
it requires major changes in architecture or new mechanical
assemblies.
VII. R ESULTS
We observed some good features of the LEDges when
compared to the AVI systems presented in section III. These
comparisons are not trivial since every inspection has its
specific requirements and we cannot test these approaches
using an identical test set.
The Table 1 shows that the less complex systems only
detect the outermost edges of the object. Those which detect
the internal edges have a complex, or very specific hardware,
and slow image acquisition. Furthermore, none of them were
implemented on embedded system. In fact most of them were
implemented in large industrial computers. Only the LEDges
is able to get the inspection time of 10ms, detecting all edges,
although implemented in low-cost embedded hardware.
Regarding the architecture based on FPGA and digital
camera, the image was acquired faster and with superior
resolution than the microcontroller-based architecture [3]. The
new inspection rate of 10ms is highlighted. We further note
that the two architectures use low-cost processors. We used
only 3% of the total available resources in the low-cost
Cyclone II FPGA.
The size of the final signature was different between implementations. The FPGA presented four times more points
than the microprocessor-based architecture. The more points
you have, the inspection will be more accurate. But for the
simple case study, both AVI systems presented hit rate of
100%. Other features are shown in Table 2 in a comparison
between implementations based on microcontroller and FPGA.
Inspection apparatus
RGGB
RGB
1
101
201
301
401
501
601
26
51
76
101
126
151
GOOD
CAMERA
SETTING
VIA I2C
COMPARISON
SIGNATURE COMPOSITION
IDENTIFICATION OF
RELEVANT TRANSITIONS
THRESHOLDING
RGGB TO RGB
CONVERSOR
+ R FILTER
IMAGE
CAPTURE
CMOS CAMERA
BAD
D5M KIT
FPGA
DISPLAY SELECTOR
SDRAM1 MEMORY CONTROLLER
SDRAM1
SDRAM2
LCD SETTING
VIA SPI
LCD DATA
INTERFACE
DE2-70 KIT
Legend:
LCD DISPLAY
LTM KIT
Fig. 4.
ACQUISITION
THRESHOLDING
SIGNATURE GENERATION
COMPARISON
LCD DISPLAYING
DATA FLOW PIPELINE
Technique
Related
works
Note
N/A
(1)
No
Wu, Boulanger
e Bischof [9]
High performance in
the segmentation
100 ms
No
66 ms
Yes
Wang et al.
[11]
Casares et al.
[4]
Simple algorithm
N/A
No
3000 ms
Yes
Wang et al.
[13]
500 ms
Yes
LEDges
Batista [3]
10 ms
Yes
High precision;
complex lighting
High precision;
complex lighting
High
performance
No (2) /
Moderate
No /
Low
No /
Low
No (2) /
Moderate
No /
Moderate
No /
High
Yes /
Very low
FPGA-based Architecture
Combinational
functions
Logic registers
Memory usage
55
146
229
67
128
162
203
990 / 68416
(1.5%)
48
94
134
32
93
135
100
636 / 68416
(0.9%)
0
19152
0
0
0
1000
12080
32232 / 1152000
(2.8%)
R EFERENCES
[1] E. Malamas, E. Petrakis, M. Zervakis, L. Petit, and J. Legat, A survey
on industrial vision systems, applications and tools, Image and Vision
Computing, vol. 21, no. 2, pp. 171188, 2003.
[2] C. Fernandez, J. Suardiaz, C. Jimenez, P. Navarro, A. Toledo, and
A. Iborra, Automated visual inspection system for the classification
of preserved vegetables, in Industrial Electronics, 2002. ISIE 2002.
Proceedings of the 2002 IEEE International Symposium on, vol. 1.
IEEE, 2002, pp. 265269.
[3] Y. Batista, LEDges: A new automatic visual inspection technique for real
time flaw detection on edges. Masters dissertation. Federal University
of Pernambuco, Brazil, 2011.
[4] T. Brosnan and D. Sun, Improving quality inspection of food products
by computer visiona review, Journal of Food Engineering, vol. 61,
no. 1, pp. 316, 2004.
[5] M. Casares, S. Velipasalar, and A. Pinto, Light-weight salient foreground detection for embedded smart cameras, Computer Vision and
Image Understanding, vol. 114, no. 11, pp. 12231237, 2010.
[6] R. Bolle, J. Connell, N. Haas, R. Mohan, and G. Taubin, Object imaging
system, May 20 1997, uS Patent 5,631,976.
[7] J. Sun, Y. Li, S. Kang, and H. Shum, Flash matting, ACM Transactions
on Graphics (TOG), vol. 25, no. 3, pp. 772778, 2006.
[8] J. Sun, S. Kang, Z. Xu, X. Tang, and H. Shum, Flash cut: Foreground
extraction with flash and no-flash image pairs, in Computer Vision and
Pattern Recognition, 2007. CVPR07. IEEE Conference on. IEEE, 2008,
pp. 18.
[9] Q. Wu, P. Boulanger, and W. Bischof, Robust real-time bi-layer video
segmentation using infrared video, in Computer and Robot Vision,
2008. CRV08. Canadian Conference on. IEEE, 2008, pp. 8794.
[10] , Bi-layer video segmentation with foreground and background
infrared illumination, in Proceeding of the 16th ACM international
conference on Multimedia. ACM, 2008, pp. 10251026.
[11] L. Wang, C. Zhang, R. Yang, and C. Zhang, Tofcut: Towards robust
real-time foreground extraction using a time-of-flight camera. Citeseer,
2010.
[12] C. Liao, J. Su, Y. Lin, and Y. Tarng, Application of the structured
illumination method for automated visual inspection of the loudspeaker
cones, Journal of Materials Processing Technology, vol. 200, no. 1-3,
pp. 5970, 2008.
[13] Z. Wang, H. Du, S. Park, and H. Xie, Three-dimensional shape
measurement with a fast and accurate approach, Applied Optics, vol. 48,
no. 6, pp. 10521061, 2009.
[14] M. Chang and S. Park, Automated scanning of dental impressions,
Computer-Aided Design, vol. 41, no. 6, pp. 404411, 2009.
[15] S. Zhang and Z. Liu, A robust, real-time ellipse detector, Pattern
Recognition, vol. 38, no. 2, pp. 273287, 2005.
[16] C. Chien, Y. Cheng, and T. Lin, Robust ellipse detection based on
hierarchical image pyramid and hough transform, JOSA A, vol. 28,
no. 4, pp. 581589, 2011.
[17] F. Duan, Y. Wang, H. Liu, and Y. Li, A machine vision inspector for
beer bottle, Engineering Applications of Artificial Intelligence, vol. 20,
no. 7, pp. 10131021, 2007.