Professional Documents
Culture Documents
Algorithm Research
Peihan WEN, Li ZHENG, Jian ZHOU, and Burton CLARKE
Abstract-Quality control plays an important and key role in However, there are few papers on flexible assembly where
manufacturing process. This paper defines a problem realizing the assembly object are flexible and there are different kinds
real-time quality control based on image processing and sensor of assembly shapes, e.g., the hydraulic hose between two
network, divides the problem into four sub-problems, as well as joints. Since the assembly shapes are not definite, some
puts forwards the overall schema to solve the problem. Then the traditional inspection methods applied in rigid assembly are
first sub-problem, Object Recognition, is focused on, its system
design pattern is presented, and main algorithms of each step not appropriate. H.S. Ip. et al. [6] treated the flexible objects
are studied. Some experiment results are given to show the good such as blood vessels as two parallel curves and recognized
performance of the proposed system for object recognition the object by detecting the parallelism. The problem of
sub-problem. parallelism detection between two curves was formulated as a
line detection problem within an affine-invariant local
I. INTRODUCTION similarity matrix computed for the two curves. Each element
Q UALITY control plays an important and key role in of this matrix gives an affine invariant measure of local
manufacturing process. With the development of parallelism for any pair of curve segments along the two
computer science and machine vision, automated on-line curves. This method is a little complicated in modeling and
quality inspection and control is more and more widely sometimes not proper under some conditions. So the method
applied in industry. Wu et al. [1] introduced the development of image segmentation comes into our sight and shows
of an automated visual inspection system for printed circuit benefits on complexity and application.
boards (PCBs). It utilized an elimination-subtraction method In this paper, we define the real-time quality control
which directly subtract the template image from the inspected problem based on image processing and sensor network,
image, and then conduct an elimination procedure to locate divide it into four sub-problems, and focus on object
defects in the PCB. Each detected defect is subsequently recognition sub-problem, present the system design pattern
classified into one of the seven defect types by three indices. and algorithms of some key steps. The rest of this paper is
Kosmopoulos et al. [2] introduced a system for automated organized as follows. Section II gives the problem definition
gap inspection that employs computer vision. It is capable of and sub-problems classification. Section III sketches out the
measuring the lateral and the range dimension of the gap overall schema of the problem. Section IV focuses on the
(width and flush, correspondingly). Lahajnar et al [3] object recognition sub-problem, gives the system design
presented a machine vision system for automated visual patter, and presents some key algorithms. Section V shows
inspection of plates of electric cookers, with the goal to
reduce~~~~~~~~~~~
lao.otadesr ossetpoutqaiy
reduce labor cost and ensure consistent product quality. some experiment results and Section VI draws the conclusion.
s
B. Problem Decomposition
The above proposed problem contains two stages: 1)
on-line quality inspection and control, 2) off-line data
analysis. Before these processes, all the assembly units
Fig. 1. Example flexible assembly image, where the black hose is the (objects) need to be recognized. Therefore, the proposed
focused object. problem can be divided into four sub-problems: object
These real industrial requirements call for computer aided recognition, quality inspection, data analysis and quality
automated inspection (CAAI). There have already been some control.
kinds of systems to perform CAAI for rigid assembly, thus The first sub-problem object recognition is to recognize all
our problem is to develop a system and corresponding main objects from the input images and record the key
algorithms to perform CAAI for flexible assemblies, e.g., as characteristic parameters; the second sub-problem quality
shown in Fig. 1, and real-time record necessary characteristic inspection is to compare key characteristic parameters of
parameters to do further statistic analysis and then give input real-time images with the standard image, inspect
quality control advice. whether the assembly expressed by the input image is correct,
and record the result as well as main comparison data; the
third sub-problem data analysis is to analyze all the recorded
Obj_ct
Recognition Input_Image data during object recognition and quality inspection, find
some statistic rules to improve quality control methods as
E g y $ 1 * Stndiardmage
object o fZ i mn image& well as quality inspection accuracy and efficiency; the final
h> ky c a -ractc
Riord rL step quality control is to apply all the real-time quality
guaranteeing methods to ensure the quality of products
On-line according to data analysis outcome.
223
(0,-) (0,1) A'
dg= d1./cosa
Fig. 4. N8 neighborhood
of these parameters will be set as the standard parameters of Fig. 5. Distance between corresponding points on both edges.
this kind of assembly, and used for corresponding inspection. constructed to describe each edge, as detailed introduced in
Then the inspection process is started, and performs Part E of this section.
real-time image capturing, on-line quality inspection, and In the final step, according to the description models of all
real-time quality control. The inspection results are real-time the edges, the characteristic parameters of the object are
reported and shown on screen, which helps workers to ensure computed, and then they are used to judge whether the
assembly quality and put right assembly faulty in time. After extracted object is the expected object, if yes, object
assembly process ends, off-line data analysis is performed recognition is accomplished; if no, there is no the expected
with all the data recorded during process running; statistic object, as detailed described in Part F of this section.
rules and corresponding quality inspection and control advice B. Image Preprocessing
later inspection process.
will be concluded forrlater
Thllbe ovcluderall inspection process.doesnot
onlyguaImage preprocessing is important and necessary step of
an
assem querallinspectyon-line,sbut
assembly quand improves qualityasorecds uulydrantae
inspection image segmentation
asond corfr remove
and pattern recognition. It
a series of random effectively
can
noise, and enhance the parts of
strteine contlysisnuouimproves quality inspection and control
interest. There two kinds of image preprocessing methods:
are
space domain techniques and frequency domain techniques.
IV. OBJECT RECOGNITION We usually use linear or nonlinear smoothing filtering (in
space domain) and low pass filtering (in frequency domain) to
Theiprped problem contains four sub-problems object
T pl and in
224
of the center pixel Xh,w to its neighbor Xh+lh,w+lw. The larger
Ah,w the more independent the center pixel.
1
Isolateathe Points f teetWo +1 e-(-K)/U (4)
1b undaties
where the parameters K and ca specify the displacement of A
No from zero and the steepness of A respectively. A reasonable
Is the real curve X-Y style estimate for K and a- is
nh nw
Yes Transin
.I I1 rlt
1
K = NAA11avv(Xh,w), (5)
Remove the reated poinfts
t
qyle h=1 w=1
0.95Qav- K
IRemoe
th eoipits e h,w) 1n4 (
+ ~~~~~~~~where
Sinothth&a
6 1
+ (
av (Xh,w) = (h,w),(h+h,W+w) (7)
Fit eakh edge points lhh=)-1+l=+
The above dav and K can be regarded as the average
distance of the center pixel to all its neighboring pixels and
the average distance of all the pixels in the whole image to its
Fig. 6. Flow chart of object edge points fitting. neighboring pixels respectively, in N8 neighborhood.
The objective function of SCCA is
G ={Xh,w h = 1,2, ...,nh, w = 1,2,_...,nW: an nhX nw c nh nw
image, where h,w are coordinates of each pixel in the JSCCA(Cr, A) = (Cri,h,W)mD,h,W (8)
image and Xhw (vector) represents the feature of the pixel i=1 h=1 w=1
(h, w); define N = nh x nw for convenience; subject to
- c: a positive integer with c > 1 representing the number of max c{f(Dk,h,w)1,
1i,h,w = f(D,h,w)/ 1<k<c (9)
clusters; where
- 0 = {fl, 02 ...J, Ocl: set of clusters with 0i representing the - Cri,h,w = Cri (Xhw) a positive number representing the
ith cluster, i E I = {1,2, ... C}; I credibility measure of feature points Xhw belonging to
- A =
(a,, a2, ..., ac): cluster center matrix, where ai (vector) fuzzy cluster Oi,
represents the feature ofthe ith cluster center for each i E I; 1 up
dL,h,w: a distance from a pixel Xh,w in the image to a cluster Cri,h,w = 2 i,h,w + 1 -SUp k,h,w); (10)
center ai in the feature space with - Cr = {Cri,h,w i E I, (h, w) E GI: a c x N matrix denoting
di,h,w = ||Xh,w - ail; (1) the credibility measure matrix;
Then, in N8 neighborhood shown in Fig. 4, where the - Mi,hhw: a positive number representing the possibility
center pixel is marked by a cross. Let Xh,w denote the center measure of feature points Xh,w belonging to fuzzy cluster
pixel, then the neighboring pixels can be denote as Xh+lhw+1w, 0i;
with Ih, lw E {-1,0,1} and (lh, lw) . (0,0). - f: a function of Di,h,w, satisfying the following equation,
The dissimilarity of the center pixel Xh,w and its neighbor 1
Xh+lh,w+1w is measured using the following Euclidean f(DL,h,w) In 99' (11)
distance l+ iDh, w
0 )(h,w),(h+lh,w+Iw)
= Xhw Xh+lh,w+lwI (2) 40
If the distance 0(h,w),(h+lh,w+Iw) is small (i.e., Xhw and - m > 1: the fuzzifier which has a great influence on the
Xh+lh,w+lw are similar), di,h,W should be greatly influenced by performance of fuzzy clustering.
di,h+lh,w+lw, otherwise, di,h,W should be largely independent
of di,h+lh,w+lw. Taking all pixels in N8 neighborhood, the The update equation of A is
dissimilarity index D1hW which measures the dissimilarity nh l ()nwl h h,w
between Xh,w and the ith cluster center ai can be given as i nh Enwfor ECIi (12)
follows: with
1
D 8hW = 1 l -- Xh,w =
1 wi1Y 1
225
(a) (b) (c) (d) (e)
(a) (b) (c) Fig. 9. Experiment result of hose 3.
226
find the re-drawn hoses are almost the same as the original
hoses, which is consistent with the recognition results and
proves the effectiveness and good performance of the
proposed system pattern and algorithms.
VI. CONCLUSION
This paper defines a real-time quality inspection and
(a) (b) (c) (d) (e) control problem based on image processing and sensor
network, and divides it into four sub-problems: object
Fig. 10. Experiment result ofhose 4. recognition, quality inspection, data analysis, and quality
control. The overall schema to solve the problem is put
1) defieaBoonelcurean t;ecv forward. Then the first sub-problem Object Recognition is
2) onec
randomlyselect s
3)
cur ent curve;
the focused on, and its system design pattern is presented. The
randomly select 100 points on te afive key steps of the object recognition system, and their main
4) calculate the distances of the selected points to the other
curve; algorithms, are studied. The system pattern will be effective
5) cv .e
Judge whether all the distances are approximately equal th this kind of flexible assembly,
for allloihsmyb whereas the details of
ifretfrdfeetknso
wihi th gie. fotrsod
within th
the algorithms may
givenerrorthresholassembly patterns or
be different
objects. The
for
median
different kinds of
error theorem for
if yes, assign the Boolean variant TRUE, and noise removing and the algorithm of parallelism judgment are
if no,ifassign the Boolean variant
n, asigntheBoolan FALSE;
ariat FLSE
very effective on hhoses, pipes, and objects of this kind.
6) select the other curve as the current curve, and perform very
step 3) -5);
effetiv on. es indojof th is kind.
This paper iS only a beginning of the automated inspection
value of tof flexible assembly problem: the problem is proposed and
7) judge the the first sub-problem is solved, the latter three sub-problems
if TRUE, the object is a hose, and
if FALSE, the object is not a hose. will be solved based on the first sub-problem and studied in
later papers.
Note that this section takes the image shown in Fig. 1 for
example to explain the system pattern and algorithms of
object recognition. For other images or objects, the system [1] W. Wu,boards
M. J. Wang, and C.-M. Lin, "Automated inspection of printed
pattern will be the same, but the details of algorithms may be
circuit through machine vision," Computers
in Industry, pp.
103-111, 1996.
different. [2] D. Kosmopoulos and T. Varvarigou, "Automated inspection of gaps on
the automobile production line through stereo vision and specular
reflection," Computers in Industry, pp. 49-63, 2001.
V. EXPERIMENT RESULTS [3] F. Lahajnar and R. Bernard, "Machine vision system for inspecting
Take four hoses in Fig. 1 as examples, some recognition electric plates," Computers in Industry, pp. 113-122, 2002.
[4] J. Derganc, B. Likar, and F. Pernus', "A machine vision system for
results as shown in Fig. 7 to Fig. 10 are given to show the measuring the eccentricity of bearings," Computers in Industry, pp.
performance of the proposed system and algorithms. In each 103-111,2003.
figure, (a) is the original image, (b) is the image after [5] M. Antonelli, S. G. Dellepiane, and M. Goccia, "Design and
(c)istheimageaftrregiongrowing
clustering segmentation, .c
clustering segmentation,
ig implementation of web-based systems for image segmentation and
CBIR," IEEE Trans. Instrumentation and Measurement, vol. 55, No. 6,
and other pretreatments, (d) is the edge image after edge pp. 1869-1877, Dec. 2006.
detection, and (e) is the recognized hose image. [6] D. Shen, H. H.S. Ip, and E. K. Teoh, "Affine invariant detection of
The recognized hose images are re-drawn using the curve perceptually parallel 3D planar curves," Pattern Recognition, vol. 33,
pp. 1909-1918, 2000.
fitting results. Checking sub-figure (e) in Fig. 7 to Fig. 10, the [7] P. Wen, L. Zheng, and J. Zhou, "Spatial Credibilistic Clustering
two edges of the black region are approximately parallel, Algorithm in Noise Image Segmentation," Proceedings of IEEE
the recognized objects are hoses. Then comparing International Conference on Industrial Engineering & Engineering
appafigurently (e)with
sub-figure sub-figure (a)
(e) with sub-figure in Fig. 77 to
(a) in 10, we'll
to Fig. I we'll AIManagement 2007, Singapore, pp. 543-547, Dec. 2-5, 2007
227