2X - Free-Form Surface Inspection Techniques State of The Art Review

Computer-Aided Design 36 (2004) 13951417
www.elsevier.com/locate/cad
Free-form surface inspection techniques state of the art review

Yadong Li, Peihua Gu*
Department of Mechanical and Manufacturing Engineering, University of Calgary, Calgary, Alberta, Canada T2N 1N4
Received 15 August 2003; received in revised form 20 February 2004; accepted 28 February 2004
Abstract
Precision inspection has been widely used in manufacturing to measure the dimensional accuracy of parts and products to meet the quality
requirements. For regular geometric features, coordinate-measuring machines (CMM) can be used effectively to assess the accuracy and
tolerances. For parts with free-form surfaces, the inspection becomes complex. Therefore, numerous researches have been carried out to
tackle both fundamental and application issues concerning free-form surface inspection. In addition to academic research, some commercial
packages have also been developed.
This paper provides a comprehensive literature review of methodologies, techniques and various processes of inspections of parts with
free-form surfaces. The specific topics cover: measurement data acquiring methods including contact and non-contact measurement
approaches; inspection planning; geometric description methods of design models or measurement data; and, the free-form surface
localization and comparison techniques, which are emphasized in this paper and mainly include the establishment of corresponding
relationship, 3D transformation solving, measurement data to design model comparison or surface to surface distance calculations. Other
issues, such as the influence factors to the localization/registration process, definition and inspection of free-form surface tolerance, and
discussions on the functions of some commercial inspection packages available on market, are also discussed.
q 2004 Elsevier Ltd. All rights reserved.
Keywords: Free-form surface; Inspection; Comparison and localization
1. Introduction
Free-form surfaces are widely used in many fields
ranging from design and manufacturing of die/mold,
patterns and models, and products in plastics, automotive
and aerospace industries to biomedical, entertainment and
geographical data processing applications. In the last several
decades, significant research and development efforts have
been made for the design and manufacturing of products/
objects consisting partially or solely of free-form surfaces.
Almost all CAD systems have the capability of designing
and modeling of free-form surfaces. Modern CNC machines
can be used to manufacture free-form surfaces. To ensure
manufacturing quality, free-form surfaces should be
inspected after they are produced.
It is a fundamental issue in product manufacturing to
determine if a manufactured object meets the design
requirements from which this object was made. For products
with regular features, such as plane and cylinder,
* Corresponding author. Tel.: 1-403-220-5770; fax: 1-403-282-8406.
E-mail address: gu@enme.ucalgary.ca (P. Gu).
0010-4485/$ - see front matter q 2004 Elsevier Ltd. All rights reserved.
doi:10.1016/j.cad.2004.02.009
the inspection techniques and equipment are mature in

industrial applications. For products with free-form surfaces,
such as marine propellers, complex sculptured surfaces are
produced with extremely high fidelity to the original design
[74]. In general, increasing demands have been observed on
the quality manufacturing of parts with complex surfaces,
which ultimately require precision measurement and inspection [4,91]. Thus, the dimensional inspection of sculptured
surfaces is essential as many products with sculptured
surfaces are designed and manufactured with a requirement
for high precision [59]. In addition, for surfaces reconstructed in reverse engineering and imaging processing, the
reconstructed surfaces also require comparison with the
reference models. The inspection of parts with free-form
surfaces is becoming increasingly critical due to increasing
requirements of higher precision and efficiency, and to the
complexity of the geometry.
In the last several decades, numerous efforts have been
reported in open literature on inspection processes, methods
and data processing techniques. As the first step of the
inspection process, geometric measurements are required.
There are two types of measurement techniques and
1396
Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417
facilities: contact measurements, normally by using a

coordinate measuring machines (CMM), and non-contact
measurements, such as laser and optical scanning. There
were several literature surveys about non-contact
measurement inspection techniques for vision-based
inspection of objects including 3D mechanical parts.
These surveys were presented by Chin and Harlow [78],
Chin [79] and Newman [85].
In this paper, a comprehensive literature review of
inspection and comparison techniques for parts with freeform surfaces is provided, which covers both contact and
non-contact measurements. For vision-based inspection,
Newman provided a review covering the literature before
1995 [85]. This paper, therefore, will concentrate on more
recent developments. Among the inspection processes, there
is another kind of visual inspection approach that mainly
deals with surface quality inspection including the inspection and detection of defects on surfaces, such as cracks,
corrosion, lightning strikes on a part or component body, or
a damaged part on a component assembly. The remote
visual inspection of an aircraft surface or skin is an example
introduced in the literature [67] and [62]. Such visual
inspection also covers defect detection for metallic workpiece surfaces [58] or wooden product surfaces, etc. This
review, however, will not cover this type of surface
inspection technique. Instead, we will concentrate on only
the inspection techniques related to geometric information
from both the measurement object and the design model.
1.1. Background
A free-form surface is a type of surface, where the shape
is not constrained by classical analytical forms and is
defined by a set of control points (as with Bezier, b-spline,
and NURBS surfaces) [61]. The characterization of freeform surfaces is given by P. J. Besla free-form surface has
a well-defined surface normal that is continuous almost
everywhere except at vertices, edges and cusps [69]. Despite
its wide applications in many fields, Campbell stated [76]
that definitions of free-form surfaces and objects are often
intuitive rather than formal.
Inspection is the process of determining if a product
(also referred to as a part, object or item) deviates from a
given set of specifications [85,71]. To carry out an
inspection of surfaces, the surface of an inspected part
can be measured by various methods such as laser/optical
scanner and CMM. The acquired data is then compared
with the design model to determine whether the surface
is out of tolerance. For a regular shaped feature such as
cylinder, its dimensions include length and diameter that
can be measured directly by instruments. The measurement values are then compared with the design
specifications, which, in this case, are the diameter and
length specifications. For a free-form surface, such
comparison does not exist. The comparison of two
free-form surfaces is done by putting these two surfaces
together in the reference position and orientation so that
the point to point comparison can be carried out.
However, the design part is in the design coordinate
system (DCS) and the real part (manufactured part) is in
the measurement coordinate system (MCS). The first step
in conducting such inspection and comparison is to bring
these two surfaces together.
Fig. 1 shows a CMM machine in (a) and the
inspection of a simple shaped part by CMM in (b). For
inspection of parts with free-form surfaces, the
relationship between DCS and MCS is complex. Fig. 2
shows the inspection of a part with complex surfaces by
Fig. 1. MCS, DCS and a part under inspection (picture modified from Ref. [7]). (a) Measurement Coordinate System (MCS); (b) Part and Design Coordinate
System (DCS).
1397
some commercial inspection packages. Finally, conclusions

are given in Section 8.
2. Measurement data acquisition methods
Fig. 2. Measurement Modeling [91].
a contact probe. In this figure, Pi is a design model point

to be measured, denoted by ri in the DCS. Due to
manufacturing errors, the design coordinate frame does
not coincide with the measurement frame. Thus, another
point Ppi is measured as shown in the figure, which is
denoted by rip in the MCS. The difference between these
two positions is Dr [91].
To compare the measurement surface with the design
model, it is essential to arrange these two surfaces in a
common coordinate system. This process is called localization. Localization refers to the determination of positions
and orientations of the DCS of a part with respect to the
MCS [92,91]. Localization is also referred to as registration
of design surface with measurement surfaces in some
literatures. In practise, locating a part is to find a rigid
body 3D coordinate transformation between the DCS and
the MCS.
1.2. Organization of the paper
The remaining sections of this paper are organized as
follows: Section 2 discusses measurement data acquisition
methods. Section 3 reviews the surface description methods
used in existing inspection and comparison approaches.
Section 4 describes the localization techniques between
measurement data and design model or between two point
sets. Specific topics include: Section 4.1 discusses the
methods for establishing the corresponding relationships
and solution techniques for 3D transformations based on
contact measurement data; Section 4.2 provides a similar
discussion but is based on non-contact measurement data;
Section 4.3 focuses on the distance calculation methods
between two surfaces; Section 4.4 deals with localization
accuracy, efficiency and robustness; Section 4.5 briefly
discusses the ICP method; Section 4.6 focuses on the
techniques of solving 3D transformation problems. Section 5
is devoted to inspection planning techniques. Section 6
discusses the techniques for inspection of free-form surface
with design datum. In Section 7, there are descriptions of
Traditional inspection of free-form surfaces has involved

highly skilled technical personnel who inspect the surface
using a number of mechanical gauges and templates [74].
This kind of process is inefficient in terms of speed and
accuracy. However, the development and use of modern
digitizing devices, such as CMM, laser/optical scanning
measurement systems and associated measuring methods,
make it possible to measure free-form surfaces quickly and
accurately.
There are generally two types of measurement data
acquiring methods: contact measurement and non-contact
measurement. Contact measurement is a measuring method
that acquires surface geometric information by physically
touching the parts using tactile sensors such as gauges and
probes. One example is CMM. Without physically contacting the part, non-contact measurement is used to acquire
surface information by using some sensing devices, such as
laser/optical scanners, X-rays or CT scans. In non-contact
measurement of parts with free-form surfaces, laser and
optical scanners are commonly used.
A review of surface digitizing techniques and classifications was presented in Ref. [55]. In that article, important
features of some existing digitizing devices and associated
measurement strategies used in surface data acquisition
were discussed. Fig. 3 shows the classification of existing
surface digitizing methods shown in this literature.
CMM has been the main tool for part validation in
manufacturing. It has high accuracy, repeatability and
reliability. The measured 3D data is used with various
algorithms to determine positions, orientations and dimensions of objects. Consequently, tolerance can be programmed, computed and verified [70]. As a contact
measuring device, CMM collects detailed dimensional
data by moving a sensing device called a probe along
work-piece surfaces (refer to Fig. 4). CMM usually acquires
data using a touch trigger probe that contacts individual
points on a work-piece. It can be used to accurately measure
objects with widely varying size and geometric configuration, and provide the relationship between the features of a
work-piece [8,60]. By applying motion control of the CMM
for non-stop surface scanning, the measuring speed can be
increased. In addition to the much higher accuracy than a
vision-based system, CMM has the advantages that it is not
affected by viewpoint or lighting conditions [60]. Thus, it
does not require clean surfaces or special illumination,
whereas a vision system always does. With the use of orientable probes, CMM can inspect surfaces that a light beam
cannot reach [3].
The disadvantages or limitations of CMMs are that the
part needs to be stationary and carefully placed and they have
1398
Fig. 3. Classification of various digitizing devices [55].
a slower measuring speed than laser and optical scanners.

This single-point measurement technique can generally
collect data at a maximum rate of about 50 , 60 points per
minute, depending on possible requirements of mechanical
fixture and the needs of programming when a different part is

inspected [1,8,16,71,70]. There is also a limitation to the size
of the part that can be measured. If an object has a very
small size (few hundreds of square millimeters), the tip of
Fig. 4. CMM measurement of a part [7].
1399
Fig. 5. Non-contact measurement illustration [7].
the tactile probe will have problems accessing part features.

Very small objects often include features with light marks or
etchings, on which even the smallest feasible mechanical tip
could not reasonably work [7]. Besides, touch probes of
CMM are also crash-prone [60].
The above-mentioned shortcomings that accompany
CMM machines can be overcome by non-contact measuring
methods such as laser scanning and optical range image
scanning. A non-contact sensor virtually allows the
simultaneous collection of all points inside its field of
view. The scanning speed of the sensor is directly
proportional to its field of view. The minimum measured
entity is not the point but the element. A non-contact
measuring sensor is a device that allows the relating of
pixels of the image or picture taken by the sensor with the
corresponding 3D points of the framed part area by basically
using a CCD camera combined with a suitable lighting
device [7]. Fig. 5 shows one of the non-contact measurement methods. There are three types of standard optical
technologies: point, stripe and area types. These are shown
in Fig. 6.
In summary, non-contact measuring methods such as
laser scanning and optical range image scanning have a
significantly higher speed of measurement (about
20000 points/s, applicable to on-line inspection [70]). This
kind of system is mandatory for the inspection of parts built
of soft material. The accuracy of recent commercial optical
systems can be as good as about 0.00100 (0.025 mm) [43,70,

44,46], which is sufficient for some applications, such as
inspection of products made from rapid prototyping process.
However, this level of precision is lower compared with the
accuracy of CMM [70,16]. Even though this represents the
most common cases, our surveying results show that some
producers claim higher accuracy for their laser scanners [45].
Based on their characteristics, both contact and noncontact measuring methods have been applied widely to
research and manufacturing fields, and will continue to be
used for a long period of time. In some applications,
solutions of CMM with a non-contact sensor have been used
or proposed for the inspection of parts with free-form
surfaces [7] and [60]. The literature [60] discussed the
integration of vision and touch sensors in a CMM controller
used for inspection tasks. This vision-probe system was
used in a cooperative interaction integration mode. Global
information generated by the vision systems was used to
guide the movement of the touch probe. Vision provided
information about the positions of part features of interest,
and then the probe was guided to the features to make actual
measurements. Similar research on integrating multiple
sensors and vision probes with CMM in order to achieve
high measuring quality and speed can also be found in some
other literature such as [83] and [88]. In Ref. [88], a system
based on the combination of laser and CMM was used to
inspect parts made of soft material. In this research, a laser
Fig. 6. Three types of standard optical technology [7].
1400
sensor replaced the probe so as to eliminate the possible

deflection of the component being measured when using
contact probes.
In acquiring free-form surface information by visual
devices, it may be necessary in many cases to acquire
several images and register them together to construct a
complete surface description [56] and [77]. This case has
been well studied in surface acquiring and reconstruction
and will not be covered in this review. This review will
mainly concentrate on the free-form surface inspection
techniques.
3. Surface description methods for localization

and comparison
The description of the 3D shapes or surfaces for
localization is a very basic task. All subsequent operations
are based on it. So in this section, we will give a brief review
about the geometric modeling methods used by existing
inspection techniques, in light of both contact and noncontact measurement data.
Earlier works in the localization described the objects by
using certain primitives such as point, line and plane, and/or
polyhedral approximations. Faugeras et al. [17] used surface
representation for recognizing and localization of 3D
objects. The surface description was in terms of points,
curves and surface patches. In detail, points were corners on
the surface or centers of symmetry, curves were either
internal boundaries or symmetry axes, surface patches were
either planes or quadrics. Detailed algorithms were given
for constructing the representation from range data. In
Ref. [22], the objects were modeled as polyhedral or
polygons. The authors assumed that the objects could be
described by sets of planar faces. Only the individual plane
equations and a polygon embedded in each face is required.
The model faces do not have to be connected and the model
does not have to be complete. To approximate the objects by
polyhedral, assumption was made to the sensed data that the
measured positions were within a known error volume and
the measured surface orientations were within a known error
zone. Gunnarson and Prinz also worked on the localization
of objects described as polyhedral [24].
Later research works applied higher order forms of
surface representation. Many approaches applied parametric
description of the surfaces, such as parametric splines,
Coons, Bezier, B-spline and NURBS surface models.
Patrikalakis et al. [63] used NURBS surface model to
represent the free-form surface patch in localization. In this
research, the positional tolerance was represented in terms
of ball-offset tolerance regions, and the tolerance region
bounding surfaces were represented by NURBS patches.
There were 10 parametric surface patches: two normal offset
surfaces, four canal surfaces and four spherical patches.
This kind of regions was for evaluating if measurement
surface was within the tolerance. In Ref. [54],
the measurement surface was described by a general

parametric surface, a NURBS patch or a number of discrete
points, while the design surface was represented in NURBS.
Jinkerson [74] discussed the localization of marine
propellers that were modeled by NURBS description.
Sahoo et al. [49] discussed the localization of 3D objects
having sculptured surface. No assumption was made to the
surfaces of the part. The surfaces could be modeled as
planner, quadric, parametric polynomial, or any implicit/
parametric form. In Ref. [59], a surface was represented in
parametric form. In Refs. [91,92], the free-form surfaces
being inspected were represented by bi-cubic parametric
Spline models based on the discrete measurement point
data. It was pointed out that the algorithms were suitable for
any smooth parametric models. In Ref. [52], the measured
free-form surfaces of sheet metal parts were approximated
by C2 continuity parametric surfaces with the specified
tolerance based on least square principle. Initially, the
control points and parameters of the surface were estimated
by a bilinear Coons surface. The parameters were further
adjusted by using bi-cubic B-spline surface. The above
surface estimation constructed an iterative process. In the
CAD-based free-form shape measurement and path planning research introduced in Ref. [2], the design model was
defined as NURBS surface patches. For localization, the
authors used the offset surface of the original CAD model
with the offset distance equals to the radius of the CMM
probe. The offset surface was obtained through three steps:
sampling points from the original CAD model; offsetting
these points along the surface normal direction; fitting
NURBS surface in least square. The fitting was iteratively
achieved within a user-defined tolerance. Approaches
discussed in Refs. [65,70 73] introduced visual inspection
of common mechanical parts. The design model in these
researches was in the format of IGES or STL, all the
surfaces on the design model were described as NURBS
patches. The measured 3D point clouds were segmented by
computing the distance between every 3D point and all of
the surfaces in the design model and by comparing some
local geometric properties between each 3D point in the
point cloud and its closest point on the design surface. The
local geometric properties used in above researches
included normal of the surface, Gaussian and mean
curvatures. The local geometric properties at an arbitrary
point P of 3D point cloud were obtained from the estimated
first and second partial derivatives at P: This was
accomplished by using a parametric second order polynomial, which was obtained by using an N X N neighborhood around P: Bispo et al. [16] investigated the inspection
of free-form surfaces by using dense range image, B-spline
model was used to describe the design model. In Ref. [48]
about optical measurement of free-form surface, a number
of point sets were captured, each of them was modeled as
B-spline surface patch. Then the overall surface was
reconstructed based on these patches, with the first patch
as the reference in the reconstruction. Huang et al. [29]
presented a method for evaluating the profile tolerance of

free-form curves and surfaces expressed in parametric form.
Input of the research could be from CMM or laser scanning.
The surface model was created by using B-spline.
In most of above approaches, the traditional surface
description methods such as B-spline/Bezier /NURBS have
been widely used in modeling of the measurement data.
However, such kind of descriptions belongs to tensor
product surface model, they may suitable for the tactile
sensing measurement data that are normally ordered, with
lower density of the volume and the surface patch can be
easily sorted to rectangular topology. While there are
limitations to the tensor product surface models in
describing measurement data from visual sensing
processes, which is normally obtained as point clouds
with dense data volume, and always with arbitrary topology.
To describe this kind of measurement data by using tensor
product surface always results in manual interaction due to
the topology issue, and/or accuracy degrading in establishing the surface model sometimes. For example, the
parameterization process for creating the tensor surface
model would cause shape deformations to the measurement
point mesh.
With the developments of non-contact measurement
methods, some recent research used the surface description
methods other than traditional approaches such as B-spline,
NURBS or others. In the approach of data acquisition and
processing for inspection of industrial parts discussed in
Ref. [47], the data of geometric position and additional
information such as the so-called covariance matrices was
stored in a multi-dimensional image array attached to the
points. Triangular meshes were used in the further
processing steps. The mesh generation algorithm discarded
those triangles that the length of any edge exceeds a distance
threshold. The experimental visual measurement system
introduced in Ref. [9] used optical sensors to acquire part
geometric information. A surface description based on
planar faces was chosen as the common domain for design
model and sensed image representation. For this, the
triangulated surfaces of the CAD model were exported
from the CAD system with triangles grouped into faces. The
measured data from different views was firstly integrated
into a single triangulated representation, and it was then
segmented into planar regions by a region-growing
algorithm. For each face, some features were calculated
such as plane equation, normal vector, 3D boundary
polygon and, etc. which were used for creating correspondence and localization process.
The non-contact measurement other than laser and
optical scanners was also carried out. In Ref. [25], the
reconstruction accuracy of industrial parts based on
measurement data from a medical CT scanner was studied.
The object surface was reconstructed by using a modified
marching cubes algorithm, and the reconstructed surface
was then approximated by triangular patches, which were
used for the comparison with design model.
1401
In comparison, the parametric representations such as

B-spline, NURBS and etc consumes less space than the
mesh representation. Once the representation model of the
surface is established, the former representation results in
faster processing in calculation of the surface information
such as differential properties, etc. and in subsequent
operations such as searching corresponding points in the
localization and comparison processes. It is normally
more convenient and accurate to study the surface
properties based on the parametric models other than the
3D mesh representation. Besides, the parametric model
establishing process normally involves higher order
expressions, for example cubic, of the surface. This also
helps reduce the influence of noise contained in the
measurement data. However, the 3D mesh representation
can directly describe the complex shaped objects
especially those with arbitrary topology and large quantity
of points very easily, which is an obvious obstacle for the
parametric representations.
Besides above mentioned surface description methods,
some researchers carried out free-form surface inspection
based on implicit representation of the surfaces. In Ref. [11],
inspections in both 2D and 3D situations were studied. The
objects in 2D images were described by their silhouettes and
then represented by 2D implicit polynomial curves. 3D data
were represented by implicit polynomial surfaces. Another
approach for evaluating free-form curves and surfaces using
implicit polynomial equations was presented in Ref. [90]. In
this research, the measurement process of the free-form
surface was driven by design model. This required the
initial implicit polynomials (IP) model of the free-form
surfaces. The points sampled from design model were fitted by
degree n IP:
X
Fn x; y; z
aijk xi yj zk 0
1
0#ijk#n
The fitting process was based on so called 3L fitting

algorithm developed in Brown University, which involved
explicit, least-square computations. The fitted IP were then
used for localization and calculation of the differences. The
quality of the implicit representation of free-form surface is
crucial to the localization and comparison in inspection,
while the description of the implicit model requires accurate
estimation of the polynomial coefficients based on the
measurement points. Fitting errors may become larger when
establish high order models, and the coefficients may not be
pose transformation invariant. Besides, in describing complex shaped 3D objects in implicit form, a number of patches
may be necessary instead of a single representation. These
are the limitations of using implicit forms.
4. Localization techniques
Traditionally, localization is achieved by presenting the
part at a desired position and orientation, using special tools,
1402
fixtures or other part presentation/orientation devices totally

dedicated for specific products. This kind of process is
usually costly, and time and effort are required to design and
manufacture new fixtures [24,49,4,13]. In recent practise,
localization has been carried out by mathematically aligning
the DCS to the measuring coordinate system by using some
initially measured data. This process allowed the use of low
precision but general purpose fixtures in flexible and small
batch manufacturing [24]. It has been formulated as the
minimization of the sum of the squared distances between
the measurement points and the design model with respect
to the transformation parameters [49,74,92,91].
In practical applications, localization can be regarded as
a two-step process: find the point point corresponding
relationship between measurement and design surfaces;
and, solve the 3D rigid transformation between these two
surfaces to bring them into a common coordinate system. In
the following paragraphs, the relevant techniques are
discussed in detail. As the inspections were based on either
contact or non-contact measurements, the review on the
localization technique is carried out from these two groups
accordingly for convenience. It should be noted that this is
not to category or distinguish the localization techniques.
Actually the method used in one group may be suitable to
another.
4.1. Localization techniques in approaches handling
contact measurement data
Traditionally, datums are measured to establish a
reference frame for the part [66]. This is known as 3 2 1
approach. Three points are measured from the first datum to
establish a plane. Two points are measured from the second
datum to establish a second plane perpendicular to the first.
Finally, one point is measured from the last datum
perpendicular to the first two [81,66]. Fig. 7 shows a
traditional 3 2 1 approach for locating DCS. This
approach has shortcomings in that the parts are required to
have plane surfaces, and the result is very sensitive to
measurement and dimensional errors. Besides, because of
the manufacturing errors on the datums, the localization
quality is sensitive to the selection of those points, there is
Fig. 7. Traditional 3 21 approach for localization ([13]).
no accurate result can be uniquely decided by choosing

these number of points.
Many approaches and methods have been developed for
localization with higher accuracy, efficiency and robustness.
Grimson and Lozano-Perez used local measurements of
positions and surface normal to identify and locate parts
modeled by polyhedral [22]. By using local constrains on
distances between faces, angles between face normal, and
angles (relative to surface normal) of vectors between
sensed points, this approach examined all hypotheses about
pairings between sensed data and object surfaces. Inconsistent pairings were discarded efficiently. Gunnarson and
Prinz [24] assumed the design surface was in the ideal
position and required an initial estimate of the actual
localization of the surface. The corresponding point pair
between measurement surface and design model was
decided by searching for the closest point. Based on the
correspondence, transformation was decided and applied to
the design model. New correspondence was created based
on the new position and localization was solved in an
iterative process, which terminated with the satisfaction of
certain preset conditions. Based on the correspondence
between measurement and design surfaces, the transformation was solved by the minimization of the squared
distance between these two surfaces.
From Refs. [54,63] it is assumed that the correspondence
of edges and vertices between the design surface and the
measurement surface was known as a priori. Localization in
this research was an iterative process. In each step,
localization departed from the position reached during
the previous step to provide an even closer position of the
measurement surface with respect to the design surface.
The transformation was solved separately by solving
rotation matrix C and translations based on the minimization of the sum of squared distances between two surfaces.
Constraints were applied to C that CCT CT C I3 ;
where I3 is the identity matrix of dimension 3, and the six
parameters (three rotational angles f; u; w and three
translation values tx ; ty ; tz ) were decided by the minimization
of following objective function OF,
OFf; u; w; tx ; ty ; tz
IX
21 JX
21
d2 rij min
i0 j0
where d 2 rij is the minimum distance between an arbitrary

point of measurement surface (at existing position) and the
corresponding member on the design model represented by
a NURBS surface. This distance was calculated by using a
modified Newton algorithm.
Similar to Refs. [54,63], Jinkerson et al. [74] used the
minimization process to find the elements of transformation.
However, in this paper, besides the six parameters used in
Refs. [54,63], the authors provided a seventh parameter,
namely the offset distance, in the minimization calculation
for solving the transformation. From the experiments, the
authors declared that the use of seven parameters resulted in
smaller RMS value after transformation compared with the

case that only six were used. The paper also extracted
the hydro-dynamically relevant features and dealt with the
impact of manufacturing errors on them. The correspondence between the measurement surface and design model
in Refs. [49,13] was decided by choosing the closest point.
This required that the initial values of the transformation be
set close to final result or the two surfaces be close enough at
the initial stage. As indicated by Sahoo and Menq, when the
part deviation from its nominal position and orientation was
small, the transformation could be recovered very rapidly.
The transformation of localization was determined by the
minimization of the objective function F;
m X
n
X
1403
arbitrary measurement point rip and its closest point (defined

as the corresponding point) ri from the design surface, the
objective function to be minimized for solving the
transformation matrix T was:
F
m
X
fi
i1
m
X
T 21 rip 2 ri T T 21 rip 2 ri
i1
For a given set of point rUp representing measured surface

and the corresponding point set rU on the design model, the
process to find the inverse of transformation matrix T21
was straightforward:
p
T 21 rU rUT rUp rUT 21
The following illustrates the procedures of the localization

process in the research:
where T is the rigid transformation matrix to be determined,

which contains the six variables, including three angles
f; u; w and three translation values tx ; ty ; tz . pij is the jth
measurement point on the ith surface patch, qij is the
corresponding point on design surface. All six variables were
obtained by solving the six non-linear equations generated
from minimization of F relative to each of these variables.
The non-linear equations were solved by the combination of
the Steepest-Descent method and the Newton-Raphson
method. As indicated in Ref. [13], solving non-linear
equations took an undesirable amount of computation time
due to the complex operations needed. The problem
worsened when a large number of measurement points
were involved. Thus, Menq and his co-workers also proposed
a modified algorithm. The new algorithm used the pseudoinverse method to determine the transformation matrix that
was approximated by a rigid body transformation. It was
claimed that the modified algorithm was 10 times faster.
For each measurement point, Balasubramanian et al. [4]
chose the closest point from the design surface as the
corresponding point. The transformation was determined by
the minimization of the objective function, which was the
sum of the squared distance between the measurement and
design surfaces. The solution was achieved through a selflearning neural network. The neural network was made to
learn the transformation in its homogenous form and the
relative transformation parameters were then determined.
According to the authors, the application of neural network
helped to reduce computational cost and enhance the
robustness in localization.
In Refs. [91 93], a simple yet effective pseudo-inverse
approach used in Refs. [24] and [13] had been modified for
localization. A significant improvement in computing
efficiency had been demonstrated. In the research, two
major operations were clearly defined: creating correspondence between the measurement and design surfaces; and
solving the transformation matrix. The correspondence
between the measurement surface and the design surface
was determined based on the closest point concept. For an
Step 1: Using initial rU from the designed surface model

and rUp from the measured surface to calculate the
first transformation matrix T21 :
Step 2: Calculate rUp (new): rUp (new) T21 rUp :
Step 3: Calculate the sum of square deviations F from the
p
two sets of point rU (new) and rU .
Step 4: Check if F is minimized within a given tolerance. If
yes, stop the process and output the result;
otherwise, continue with the following steps.
Step 5: Using the transformed points rUp (new) of measured
surface and rU : from the designed surface to find
next approximation of T21 :
Step 6: Go to Step 2.
lT 21 pij 2 qin l2
i1 j1
The above algorithm was extended to localization with

design datum in Ref. [91]. This case is reviewed in Section 6.
Li et al. [95] introduced a machined geometry estimation
and inspection simulation system for 3D parts with complex
shapes. For the inspection function, this system simulated
the CMM measurement by shooting a number of rays to the
estimated 3D free-form surface; the measurement points
were those intersection points between the rays and the
estimated surface. The localization techniques developed in
Refs. [91,92] were applied.
Pahk [26] proposed a precision inspection method for
parts with very thin and sharply curved features. The
localization procedures of this method had rough and fine
alignments. The rough alignment was based on the
conventional 6 points probing around the clear-cut surfaces
of a reference square block. The fine alignment used
iterative least-square techniques based on the measurement
data of the pressure/suction surfaces. The rough alignment
in this research is actually a 3-2-1 approach.
For localization of measured surfaces to the design
model in the free-form surfaces shape error evaluation
process of sheet metal parts introduced in Ref. [52], it was
assumed that the measurement surface was positioned to
roughly fit to the design model, which was guaranteed by
manual alignment. In this research, only rough localization
1404
of those interested regions was required at the initial state.

The detailed correspondence was based on the point
matching between the measurement surface and design
model, which was based on the evaluation of the curvature
values at those points.
Ainsworth [30] discussed free-form surface inspection
using a CMM and used ICP (Iterative Closest Point) for
determining transformation. To provide a good estimation
of transformation for cases that the design model and
measured part were initially grossly misaligned, manual
assistance was applied to find a number of corresponding
points from the measurement surface and the design model
to generate a rough alignment. ICP was then applied to the
subsequent registration.
The implicit polynomial based free-form surfaces
comparison was reported in Ref. [90]. This approach
calculated the errors between the measurement points and
the design model in two steps: fit IP using the points
sampled from design model, and then compute the
perpendicular distance from the implicit polynomial
model to measurement points.
It is found that, for the purpose of localization,
corresponding points between measurement surface and
design model need to be decided. The existing techniques
search the corresponding points mainly based on the idea of
choosing closest point between these two surfaces. This
imposes limitation to the localization process: the localization can work properly only when it starts from certain
initial condition that the measurement surface and design
model are close enough in both 3D position and orientation.
To satisfy this requirement, some measures such as manual
assistance, and prior assumption of the transformation or a
number of estimations of preparing transformation is
required. Sometimes design datums are used for the
alignment for this purpose. All these result in lower
efficiency of the localization process, and unique solution
may not be guaranteed.
The automated visual inspection system introduced in

Ref. [84] for detecting defects of castings by using range
images was applicable to a class of objects containing planar
and/or quadric surfaces. Localization was based on the
assumption that the expected orientation of the parts was
known. Therefore, the system only needed to solve three
translation equations and one rotational equation.
Methods introduced in Refs. [65,70 73] conducted
visual inspections of common mechanical parts and made
comparisons between measurement surfaces and their CAD
models. The corresponding relationship was established
using a modified ICP process, which selected the corresponding point based on the evaluation of distance between
potential corresponding points and the surface curvature
values (Gaussian curvature and mean curvature) at these
points. Transformation matrix T was found by using
quaternion representation, which was used or discussed in
Refs. [17,5,68]. The geometric models were described in the
NURBS format. Fig. 8 shows the inspection process.
The rotation matrix Rq was expressed as:
2
3
q20 q21 2q22 2q23 2q1 q2 2q0 q3 2q1 q3 q0 q2
6
7
6
7
Rq 6 2q1 q2 q0 q3 q20 q22 2q21 2q23 2q2 q3 q0 q1 7
4
5
2q1 q3 2q0 q2 2q2 q3 q0 q1 q20 q23 2q21 2q22
6
where qq0 ;q1 ;q2 ;q3 ; q0 $0; q20 q21 q22 q23 1: The
optimal rotation matrix was given by the unit eigenvector
corresponding to the largest eigen-value of the 4 4 matrix,
whose components were generated by the covariance matrix
between the two point sets.
The inspection method based on implicit polynomial
expression introduced in Ref. [11] assumed that
4.2. Localization techniques in approaches handling

non-contact measurement data
Compared with the contact measurement, non-contact
measurement normally generates a larger volume of
measurement points. The measurement points may not be
organized and may contain more than one geometric
primitive. To establish the correspondence between the
measurement data and the design model, segmentation is
normally carried out to find possible primitives.
Bispo [16] investigated localization or the matching of
acquired free-form surface image data with the design
model. No salient features (such as planar regions) that
could help guide the registration were assumed. The
matching was based solely on the 3D points with an
estimation of the pose alignment. Studies on the accuracy of
the alignment and relative influence factors in this research
are covered in Section 4.4.
Fig. 8. Block diagram of the inspection system [65].
the alignment between the measurement object and the

design model was done beforehand. The inspection activity
was then to model the template of the design model by an
implicit polynomial. Edges of the image of the measurement
object were extracted. Each edge point was tested if it was
inside tolerance values.
Pottmann et al. [27] made surface inspections and
comparisons by localizing 3D point clouds from laser
scanning to its CAD model, based on a modified ICP
method. The localization was an iterative process, which
was very similar to the ICP process. Suppose xi is one of the
points from the measured 3D point cloud, yi is the possible
corresponding point on the design model. In each iteration
of the localization process, instead of moving xi towards yi
which is the case of the standard ICP, the modified ICP
method moved xi towards the tangent plane of the design
surface at yi : It was claimed that the modified approach
converged much faster than the standard ICP approach. As
indicated by the author, for low curvature surface regions,
this difference on convergence was more obvious.
Fan and Tsai [48] used two CCD cameras and a laser
diode to acquire a number of free-form surface patches and
reconstructed the complete part surface based on these
image data. The shape error of the matching image of the
free-form surface was defined as the maximum value of the
nearest distance between the different patches. The initial
localization between different patches was carried out based
on human-computer interactions. Then the detailed localization was solved based on the minimization of the
objective function, which was the sum of the squared
distance between the two surface patches to be studied.
In a visual inspection system for stampings with freeform surfaces introduced in Ref. [75], localization was
actually a 2D operation for solving two translation variables
and one rotational variable. The translation was the planar
motion between the centers of the measured stamping image
and the reference model.
Guehring [47] treated two processes of registration and
localization. Registration of multiple views of measurements was for surface reconstruction. Localization was to
align the reconstructed surface to the design model, for
comparison between those two surfaces. All these registrations were based on a modified ICP algorithm. The
corresponding points were defined as the point pair that was
close in both distance and normal directions:
kd 2 sk , td and ns nd . ta
where d was a point from the design model; s was the

measurement point that potentially corresponds to point
d; nd and ns were surface normal at these two points; td
was distance threshold; and ts the cosine of an angular
threshold. Two types of correspondences were discarded:
corresponding points lying on the boundary and corresponding points that were too far away. For solving the
transformation, the rotation matrix was expressed in unit
quaternion, and the transformation was estimated based on
1405
the minimization of the covariance-weighted sum of leastsquare of the differences between corresponding points.
The visual inspection introduced in Ref. [80] dealt with
the robotic surface inspection of a space platform. In the
space environment, there are larger repeatability problems
due to robotic arm flexibility and object location changes
caused by thermal expansion and structural flexibility. The
reference image and inspected image were registered by
using a Gauss-Newton iterative method. The objective was
to find a suitable transformation of the reference image so
that the residual between the inspection data and transformed reference image was minimized. The Gauss-Newton
algorithm solved this non-linear least-squares problem via
an iterative process, stopping when the least-squares
residual dropped below a pre-defined threshold.
Delingette et al. [25] reported the comparison between
reconstructed surface mesh with the design mesh. The
correspondence and registration between these two meshes
were based on an ICP approach that iteratively selected the
closest points as the corresponding points and estimated the
best transformation, until a displacement threshold was
reached. To take into account the outliners, an algorithm
was implemented to remove the vertices that were located
too far away.
Similar to those approaches handling contact measurement point data, most of the methods for the localization of
non-contact measurement point data discussed above also
decided corresponding points by searching closest point.
ICP and/or modified ICP were commonly applied in these
approaches. The same limitation discussed in previous
Section 4.1 also exists to these approaches in processing
non-contact measurement points.
Different from above approaches that used points directly
for deciding the correspondence, some methods selected
surface primitives or features and used these items to create
the corresponding relationship. Faugeras [17] represented
the surface in primitives and carried out localization in two
tasks: a recognition task to produce a list of corresponding
primitives from measurement range image and design
model; and, a locating task to calculate the transformation
to bring the design model to acquired range image. The first
task was implemented by using the tree search method. The
transformation was decided by using quaternion. In
selecting the primitives for localization, the authors
recommended that line primitives should not be parallel
and planes should be independent. The minimum number of
primitives required for localization was listed in the
following Table 1. The localization mainly depended on
the existence of planar regions in the object being matched.
Based on the similar localization algorithms of [17], a
vision-based inspection approach presented in Ref. [1]
decided the correspondence by matching the segmented
primitives from the design model and measured image data.
In this research, the objects contained planar, cylindrical or
spherical faces, but only planar faces were used for
localization. The estimation of transformation was treated
1406
Table 1
Minimum number of primitives needed to establish the transformation [17]
Points
Lines
Planes
Translation
Rotation
3
2
3
3
2
2
as a least-square minimization problem. If the estimation of

the rotational elements of the transformation matrix was not
a straightforward minimization, quaternion might be used.
Improvements were made in this approach based on the
method introduced in Ref. [17]. However, it has shortcoming that only planer primitives were used for the
localization, this requires a number of planer primitives on
the object studied for the localization.
The correspondence created between the design model
and measurement data in Ref. [9] was based on the
constrained tree search approach. In establishing the
correspondence, the process started with one matching
pair between the design model and the measurement image.
Once each possible pair was identified, the search went to
the next level. The search was a recursive process. In order
to control the search time, constraints were used to bind the
branching in the tree. So for each measurement feature, only
a subset of design model features was selected as possible
matches. During the search, the skipping of features was
allowed; if no correspondence was found for a certain
measurement feature, this feature was removed from current
matching path and the search continued. Once the
correspondence was established, the rigid body transformation between the measured data and the design model was
estimated.
Different from the approaches by using directly the
surface points in the corresponding and localization process,
the methods based on using attracted surface primitives or
features have the advantage of less user assistance
involvement. This is meaningful for improving efficiency
and automation level. However, the reliability and accuracy
of the primitive or feature extraction are crucial.
4.3. Comparison between the measurement surface
and the design model
The calculation of distance between the measurement
surface and the design model is an essential and critical
operation in the localization process. For most of the
existing approaches, localization is an iterative process and
the calculation of the distance between two surfaces is
required in every iteration. The distance calculation is the
main cost of the localization time. The comparison between
the measurement data and the design model is actually to
calculate the distance between them.
In calculating the distance between the measurement
point to the design surface, represented by parametric
bi-cubic splines surface patch, Gunnarson and Prinz [24]

searched for the nearest point from the design surface for
each measurement point, and approximated the design
surface at this point with a plane. The distance between the
measurement point and the design model was calculated as
the point-to-plane distance.
To compare the measured points to design model,
Patrikalakis [63] firstly selected the maximum among all
minimum distances from the localized measurement surface
points to the design model. Then, this maximum distance
was used to verify if the measurement surface was within
the pre-defined bounding surfaces of the tolerance region. If
the measurement surface equation was known, the verification was reduced to the interference detection between the
localized measurement surface and the bounding surfaces of
the tolerance region. A similar idea was used in Ref. [54].
Sahoo and Menq [49,13] discussed two methods for
distance calculation based on the complexity of the part and
the type of surface representation. The first method was
Orthogonal Euclidian Distance, which was suitable for
surfaces represented in either parametric or implicit form.
For a general bi-variate surface expressed in terms of
parameters u and v; the squared distance could be calculated
using:
D xmi 2 x2 ymi 2 y2 zmi 2 z2
where xmi ; ymi ; zmi is an arbitrary measurement point,

x; y; z is the corresponding point on design model. By
solving the following non-linear equations:
D
x
y
z
x2xmi y2ymi z2zmi 0
u
u
u
u
D
x
y
z
x2xmi y2ymi z2zmi 0
v
v
v
v
10
The parameters u;v can be found to determine the

distance. The second method was Algebraic Distance,
which was suitable for surfaces represented in implicit form.
According to the authors, this method worked well for
surfaces of planar, quadric and lower order parametric
polynomials. For higher order surfaces, this method became
computationally expensive. Therefore, the Orthogonal
Euclidian Distance method was recommended for higher
order surfaces [49]. This method was also used in Ref. [4]
for calculating the shortest distance between the measurement points and the design surface. The solution was
achieved by solving non-linear equations.
In Refs. [91 93], for an arbitrary measurement point rip ;
its closest point (defined as the corresponding point) ri from
the design surface, presented in parametric form, was
defined by:
r i
0
u
r
T 21 rip 2 ri T i 0
v
T 21 rip 2 ri T
11
12
where u and v were design surface parameters; T was

transformation matrix; and a modified Powell hybrid
approach was used to solve the non-linear equations.
Pahk and Ahn [26] evaluated the difference between the
measurement points and the design model in such a way
that, the correspondence was decided by the closest point
concept at first. Then, for every measurement point, the
corresponding point on the design surface was calculated
based on an iterative subdivision algorithm. Finally, the
deviations were obtained. For segmentation of 3D measurement data, distance from 3D measurement point to a
NURBS surface was used in Refs. [65,70 73]. The distance
of a point to a NURBS surface was computed as finding a
point on the parametric surface such that the distance
between the 3D measurement point ~r and the point on
the design surface was minimal in the perpendicular
direction to the tangent plane at the point from the design
surface (refer to Fig. 9). The function to be minimized was:
minu0;v0 k~r 2 ~su; vk2 ; which could be solved by an iterative
process.
Calculation of the difference between the measurement
data and the design model in Ref. [52] was divided into two
categories: local evaluation and global evaluation. The local
evaluation was the comparison between points based on the
value of their Extended Gaussian Curvature L defined as:
L lk01 2 k1 k02 2 k2 l
13
where k01 and k02 are the principle curvatures at the

measurement point; while k1 and k2 denote the principle
curvatures of the corresponding design point. A matching
rate function was designed to evaluate the local errors. The
global evaluation was to extract and evaluate the surface
features such as a bend or twist. The relationship between
the aggregate normal vectors of the surface features was
decomposed into a bent angle and a twisted angle. The
differences of bent and twisted angles constituted the global
evaluation results.
Fig. 9. Point to NURBS surface distance [65].
1407
In Ref. [48], the nearest distance between two sets of

point clouds represented in B-spline surfaces was studied.
The nearest distance was defined as the intersection between
the surface normal from one of the two patches to another. A
so-called direct method was used, which was a minimization process based on the Newton-Raphson method.
Huang and Xiong [29] treated the calculation of distance
from the measurement points to the design surface as
solving a non-linear optimization problem. A laconic
algorithm was used to solve it. The calculation was an
iterative optimization process that converged to the solution
linearly.
Comparison between reconstructed mesh obtained from
CT scanning and the design model was made in Ref. [25]
based on several different distance criteria for evaluating
the shape similarity. The distance was between the vertex
of the design model and its closest point on the
reconstructed model. The distance criteria were media
distance, maximum signed distance, and maximum and
median distance at the edges and the corner vertices. A set
of parameters relative to the processes from the 3D model
digitization to reconstruction of a 3D mesh were evaluated
for their impact on reconstruction accuracy. It was
concluded that this methodology was well suited for the
inspection of smoothly curved mechanical parts. However,
the resolution of the CT used in the research may create
some limitations for certain applications and a more
accurate scanning device, such as industrial CT scanner,
may be necessary for some cases.
In summary, the difference calculation between measurement surface and design model is actually the calculation of
the distance between measurement points to design surface,
which can be implemented by calculating the point-plane
distance such as the methods discussed in Refs. [24,65], etc.
or by directly calculating the point-point distance between
closest point pair/corresponding points between those two
surfaces. Experiment results show that the point plane
distance calculation is faster than the point point distance
calculation. For surface expressed in parametric model, the
problem of searching the plane or closest point from design
model can be solved by calculation or deciding the proper
parameter value. While the required parameter values can
be obtained by solving non-linear equations such as the
approaches discussed in Refs. [13,49,4,93], etc. or by some
other methods such as the subdivision method introduced in
Ref. [26].
For above two different approaches, namely method
based on solving non-linear equations and methods of
subdivision, the first one is normally faster, but may not
guarantee solution for all conditions, for example, for some
boundary points. The second approach has slower calculation speed, especially when the required subdivision step
is very small. The advantage of this method is that it can
always provide reasonable solution.
1408
4.4. Measurement point selection, accuracy, efficiency

and robustness in localization
The selection of measurement points is an important
issue for inspection. In CMM measurement of free-form
surfaces, the selection and quantity of measurement points
have an important effect on inspection speed, localization
and the comparative processes. It is desirable to have an
inspection method that requires the fewest measurement
points but provides sufficient information for localization
and comparison between measurement surface and design
model.
A conventional way to define discrete measurement
points on a measured entity is to define a 2D rectangular grid
(parallel to one of the primary planes) within the 3D
model environment. Once the grid has been positioned and
the number of points on the grid specified, the grid points
are then projected onto the model surface, each point
of intersection between the projection lines and the model
will specify a measurement point. Fig. 10 shows such a
process [2]. The disadvantage of this method is that it is not
suitable for complex shapes, such as free-form surfaces with
high curvature.
In selecting the measurement points for localizing
polynomials, Gunnarson and Prinz [24] suggested measuring those facets that have the greatest angular separation.
The points should also be spread over the surface to
facilitate accurate orientation estimation. Sahoo et al. [49]
suggested that one should try to choose as many points on
the planar or less complicated surfaces for faster processing
speed. In the localization process by visual measurement
studied in Ref. [16], the sampled points at multi-scale from
both the design model and measurement point set were used
for localization based on ICP. After the convergence using
coarser grids of data points, better accuracy was claimed by
running an extra iteration of the ICP algorithm with more
Fig. 10. Conventional method for generating measurement points [2].
data points. Ainsworth [2] decided to choose the measurement point based on adaptive subdivision sampling of the
design model. Several surface sampling criteria were
identified and implemented, including: uniform sampling
in the u and v parameter directions; chord height criterion;
minimum sample density criterion; and, parameterizationbased sampling criterion. These criteria can be used
individually or in combination. Research has also been
carried out regarding measurement point selection and
quantity, and their influences on the accuracy, efficiency and
robustness of the localization process. In Ref. [13], a
statistical analysis was used to determine the number of
measurement points for a surface profile based on the
tolerance specification and manufacturing accuracy. The
authors demonstrated that the number of measurement
points depended not only on the design specifications but
also on various factors in design and manufacturing. Design
specifications and manufacturing information can be
brought together to determine the desired inspection
strategy via statistical analysis. For localization based on
the iterative techniques, accuracy has a direct relationship
with the quantity of measurement points. As observed in
Refs. [13,96,81], the more the measurement points, the
more accurately the localization can be reached, or the more
the registration accuracy can be greatly increased [2]. It was
also indicated in Ref. [4] that the average transformation
error would be reduced as the number of points increased.
For dense data, the transformation error would be very small
compared to the average orthogonal error. Based on the
experimental results, Huang et al. [91] concluded that
increasing the number of measurement points could
improve the accuracy in localization.
It should be noted that increasing the point quantity
might not always give positive influence to the localization
process. More points results heaver distance calculation
between measurement point to design model, this is obvious
for many of the localization process that the corresponding
searching and transformation solving is solved in an
iterative process. Besides, noise contained in the points
always cause negative influence to the accuracy of the
correspondence, while for most of the localization
approaches, the transformation information is obtained
based on the corresponding point sets from both measurement and design surfaces. Adding more points with higher
lever of noise will worsen the localization accuracy instead
of improving. Therefore, judgement measures need to be
considered, such as what kind of points helps improve the
localization accuracy and what kind of points should be
avoided.
The accuracy, efficiency and robustness of the localization process are also influenced by many other factors
besides the point quantity. Some of the researchers
provided suggestions and comments based on their
experiments. Gunnarson and Prinz [24] expected, based
on the localization of objects represented in polyhedra, that
localization inaccuracy was on the same order as
the combined surface roughness and sensor uncertainty.

Sahoo [49] concluded that the speed of the minimization
algorithm depended on the rate at which distance could be
calculated in the minimization process. In applying the
localization algorithm to actual objects, points on the planar
or less complicated surfaces were preferred because the
distance calculation for these surfaces was faster.
When measuring a surface using CMM, the selection of
measurement points is arbitrary. Singularity might occur.
Dimensional errors of the selected measurement points have
a direct influence on the accuracy of the transformation. In
Refs. [49,13], the robustness of the localization algorithm,
as related to the above influence factors, was discussed. One
way to measure the sensitivity of the objective function F to
a small variation in the six transformation variables was
introduced by examining the singular values of a sensitivity
matrix. Moreover, a sensitive measure, based on the
smallest singular value, was proposed and used to estimate
the transformation errors.
For inspection accuracy when using a laser scanning
technique, Marshall [1] concluded from experiments that
the depth information obtained from edges was less accurate
than face data. In Ref. [16], a number of experiments had
also been conducted on the transformation accuracy for
localization/registration in visual inspection. These experiments were run for different values of rotation and
translation, and for different levels of noise. The results
showed that the maximum accuracy that could be achieved
for a given level of noise corresponded to the combined
effects of the sensor, modeling errors and the number of
matched points. The error in the translation estimation
decreased almost inversely with the number of matched
points. The initial translation estimation had influence on
the final registration accuracy. Based on their and others
experimental results, they concluded that when the
measurement data set covered a reasonable portion of the
design model, the ICP performance was reasonably
insensitive to the original translation estimation. However,
final accuracy might change considerably and the situation
even worsened when the measurement data covered only a
small portion of the design model. Prieto et al. [73]
indicated that the size of defects, which could be detected,
depended in a significant way on the resolution of the
sensor. The authors experimented with a 3D measurement
sensor that had the capability of detecting as small a defect
as 0.05 mm of depth.
4.5. Brief discussion on the iterative closest point (ICP)
method
In almost all of the localization approaches introduced in
Sections 4.1 and Section 4.2, the corresponding relationship
between two surfaces was determined by choosing the
closest point set. As an important and essential step of
localization, the solving of the 3D transformation matrix
was closely related to and depended on the corresponding
1409
relationships. If the selection of the corresponding points is

determined by choosing closest points, as practised in the
above approaches, new corresponding relationships can
again be established after applying the transformation to the
measurement surface. Therefore, the operations of solving
the 3D transformation matrix T and the finding of the
corresponding relationships constitute an iterative localization process. This iterative process has been called ICP
[68] [15] and has been commonly used by many of the
aforementioned approaches.
ICP tries to find the minimum difference value between
two surfaces. However, the parameter space explored by
ICP can contain many minima [76]. Therefore, measures
need to be taken to avoid converging to a non-optimal
solution. In actual applications, the surfaces being compared
need to be located close enough before applying ICP.
Preprocessing or user interaction is required for this
purpose. Some researchers used a number of initial
estimations of registration or localization for finding the
global solution [72].
The ICP method has also been commonly used in surface
reconstruction and surface patches registration processes.
As mentioned in Refs. [76,15], the ICP approach required a
good initial estimation of the transformation that was
reasonably close enough to the true value in order to
converge to a global minimization. Some researchers
provided this initial estimation by using a mechanical
device, such as a turn table [56]. There were other methods
that solved this problem: initial guesses of the transformation matrix T; assuming a priori knowledge of an estimated
pose that aligned the different range images and approximated one of the surfaces being matched using its tangent
planes [94]; or, modification of the ICP process to find a
better initial 3D transformation [23,18,10].
Some researchers also used features either alone or
together with point positions. Chua [12] used principle
curvatures and direction constrains. An approach called
ICPIF has recently been proposed that used shape features in
conjunction with point positions in ICP to register range
images Chua [19]. The weighted invariant features,
including principle curvatures, moment invariants and
spherical harmonics invariants, together with the positional
information were combined to construct a distance function
for choosing the closest point in registration.
Another potential shortcoming of the traditional ICPbased registration is that the transformation estimation is
done by calculating mean square (MS) distance. This was not
robust to outliner points generated either by noise or by the
presence of other parts in the scene [73]. Some methods were
proposed to overcome this problem. Masuda [82] estimated
the transformation between two range images in a robust way
by fusing the ICP algorithm with random sampling and leastmedian of the square (LMS) estimation. Registration can
then be achieved with a high degree of robustness.
In general, although ICP and its variations are the
dominant methods for localization, this approach has an
1410
obvious limitation in that two surfaces in localization must

be initially located close enough in order to determine the
corresponding points. In practical applications, the two
surfaces may have arbitrary positions and orientations in 3D
space, which makes the determination of the corresponding
relationships difficult.
4.6. Discussion on solving 3D transformation
In previous Sections 4.1 and Section 4.2, we mentioned
some techniques for solving the rigid body transformation
for localization process, such as the method of solving nonlinear equations introduced in Refs. [49,80] and the pseudoinverse method discussed in Ref. [13]; the minimization
method illustrated in Refs. [54,63,74]; the minimization
approached solved by a self-learning neural network by
2
6
6
6
N6
6
6
4
empirical, graphical and numerical methods, which could

give an approximate solution leading to a better, but still
imperfect, answer. The iterative process requires an initial
estimation of the transformation.
Horn reviewed analytical methodologies that could
provide more efficient, robust and reliable solutions. He
proposed a closed-form solution for this problem [5]. In this
method, the unit quaternions were used to represent
rotations. The translation offset was the difference between
the centroid of the coordinates in one system and the rotated
centroid of the coordinates in another system. The unit
quaternion representing the rotation was the eigenvector
associated with the most positive eigen-value of a
symmetric 4 4 matrix N: The elements of this matrix
were simple combinations of sums of products of corresponding coordinates of the points:
Sxx Syy Szz
Syz 2 Szy
Szx 2 Sxz
Sxy 2 Syx
Syz 2 Szy
Sxx 2 Syy 2 Szz
Sxy Syx
Szx Sxz
Szx 2 Sxz
Sxy Syx
2Sxx Syy 2 Szz
Syz Szy
Sxy 2 Syx
Szx Sxz
Syz Szy
2Sxx 2 Syy Szz
Ref. [4]; the modified pseudo-inverse approach developed

in Refs. [92,93]; the method of using quaternion in Refs.
[17,65,73], etc. For above approaches, the method of
solving non-linear equations normally is costly regarding
the computation time, especially when more points are
involved. The pseudo-inverse approach and the selflearning neural network assisted method improve the
processing speed, while the modified pseudo-inverse further
reduced the processing cost in solving the transformation.
The methods by using quaternion or unit quaternion have
obvious advantages and this will be covered below together
with some other techniques.
Solving 3D rigid body transformation is also a critical
issue in image registration, surface reconstruction and
robotics. There is a common problem of finding the 3D
transformation between two sets of corresponding points.
For example, to register two different image patches or
surface patches together for constructing a complete image
or surface, it is necessary to apply 3D transformation to one
of the image or surface patches to align it to another. In
robotics, measurements in a camera coordinate system need
to be transformed to coordinates in a system attached to a
manipulator. Finding the proper 3D transformation is one of
the major tasks. The solution methodology has a direct
influence on registration accuracy and speed.
Since measurement always contains errors, it is not
possible to find a transformation that maps the measured
coordinates of points in one system exactly to those in
another system. In practice, the transformation is usually
determined by minimizing the sum of squares of residual
errors. There were some iterative methods, such as
3
7
7
7
7
7
7
5
14
P
P
where Sxx ni1 x0l;i x0r;i ; Sxy ni1 x0l;i y0r;i and so on; and,
x0l;i ; y0l;i ; z0l;i and x0r;i ; y0r;i ; z0r;i are the coordinates of the
corresponding point pair from those two set of points. To
find the eigen-values, a quartic equation has to be solved
whose coefficients are sums of products of the elements of
the matrix. The minimum number of the corresponding
point pairs required to determine the transformation is three,
and the points should be non-collinear.
Arun [53] introduced a closed-form solution to leastsquare minimization in order to estimate the transformation
between two corresponding point sets. The transformation
was represented by an orthonormal rotation matrix and a
translation vector. The rotation parameters were estimated
by a procedure based on the Singular Value Decomposition
(SVD) of a 3 3 matrix. Horn et al. [6] proposed another
closed-form solution to solve the least-square problem in
estimating the transformation based on the use of an
orthonormal matrix. The translation was represented as the
difference between the centroid of the coordinates in one
system and the rotated centroid of the coordinates in another
system; and, the rotation was represented by using
orthonormal matrices. This method depended on eigenvalue eigenvector decomposition of a 3 3 matrix and so
required the solution of a cubic equation. Upon comparing
the methods introduced in Refs. [5,6], the method based on
unit quaternions was more elegant, according to the authors.
Walker [87] presented an algorithm for estimating the
3D transformation using dual quaternions. The problem
was formulated as a dual number quaternion optimization
problem about a single cost function associated with
the sum of the orientation and position errors. There
are two parts of a dual quaternion q^ : q^ r 1s; where

sinu=2n
d=2cosu=2n sinu=2p n
r
; s
:
cosu=2
2d=2sinu=2
Based on the experiments, Lorusso [57] made a
comparative analysis on the four closed-form solutions
listed above, from Refs. [87,53,5,6], respectively. The
comparison was based on three issues. The first issue was
the accuracy of the algorithms for solving the transformation using coordinates of the corresponding points with an
increasing amount of noise. The second issue was about the
stability of the algorithms when the original 3D point sets
degenerated into planes, lines and simple points. The third
issue was the relative efficiency in terms of actual
processing time by each algorithm. The comparison result
was summarized in Table 2.
A closed-form solution for the least-square estimation of
transformation was also introduced in Ref. [86]. This
method was based on SVD and could be regarded as the
extension or refinement of the SVD approach introduced in
Ref. [53]. This new SVD method could provide a solution of
the transformation even if the input data was corrupted. The
method was applicable to any dimension. In application,
Kanatani [51] applied the SVD method from Ref. [53], the
unit quaternion approach introduced in Ref. [5] and the
orthonormal matrix method presented in Ref. [6] in a refined
manner. The author tried to solve the 3D transformation
based on available correspondence in a concise way based
on the so-called optimal resolution of degenerate rotation. A
covariance matrix of rotation fitting was defined, and the
statistical analysis for the fitting error based on the
covariance matrixes was conducted.
Least-square minimization was used by all the solutions
listed above to estimate the transformation. It has become
the basic and main stream in solving this type of problem.
However, this may not be true in some researches. In
Ref. [64], an optimal rotation estimation algorithm was
proposed. This method was different from other approaches,
which was not based on least squares minimization. This
method first created a theoretical accuracy bound, and then
estimated the rotation by using an optimal solution that
attained that accuracy bound. This optimal solution was
non-linear and it was processed by using quaternions and
renormalization techniques. No translation was handled in
this research. The noise considered by this approach
included anisotropic and inhomogeneous noises. Based on
1411
experiments, the authors claimed that this method was

superior over least square method.
5. Inspection planning techniques

5.1. CMM measurement planning
The major mechanical components of a CMM machine
consist of a CMM column, probe stylus and probe tip.
During inspection, the probe moves at high speed. Collision
can easily damage one of the CMM components. The
human inspector can monitor the inspection and modify the
inspection path manually to avoid collisions. This is,
however, time consuming and prone to error. Therefore,
automated planning for CMM inspection is an important
research topic.
A hierarchical planning system was proposed in Ref. [59]
for automated dimensional inspection of complex shaped
parts with free-form surfaces using CMM. The system could
detect collisions of the probe tip, the stylus and the column.
The collision-free inspection path was automatically
generated, with the automated selection and modification
of the probe angle, based on a local accessibility analysis of
a given feature. The hierarchical planning and heuristic
matching has also been discussed in Ref. [28]. The authors
concentrated on the collision-free inspection path when
using the CMM. The goal of the research was to automate
the process for the dimensional inspection of dies and molds
with complex surfaces. Issues of CMM path planning and
collision detection against probe tip, probe stylus and CMM
column trajectories were investigated.
A method for automatic generation of high-level
inspection plans of mechanical parts was proposed in
Ref. [3]. A high-level plan is a collection of the setups of the
part on the CMM, the features to be inspected in each setup,
the probes, and the probe orientations to be used for each
feature. A general method for computing accessibility was
presented, which is one of the most important geometric
constraints for inspection planning.
Spitz [81] presented the development of a fully
automated dimensional inspection planner for CMMs. The
planner first generated a high-level plan that specified how
to set up the part on the CMM table. This plan was then
expanded to include detailed path plans and ultimately a
program for driving the CMM. The planner was
implemented and included an accessibility analysis module,
Table 2
Comparison of four closed-form solutions for solving 3D transformation [58]
SVD
Unit quaternion
Orthonomal matrix
Dual quaternion
Accuracy
Stability
Efficiency
Comments
Reference
Almost same
Almost same
Almost same
Almost same
Stable
Stable
Less stable
Poor
Good for small data size

Good for large data size
Good for accuracy and stability

Better than SVD for small data sets
Good for large data sets
[53]
[5]
[6]
[87]
1412
a high-level planner, a plan validator (through collision

detection), a simulator, and a path planner.
An object-oriented generative inspection planner was
introduced in Ref. [66], which included an inspection
process planner and an inspection path planner. Inspection
related information, such as dimensions, tolerances and
geometric items, was retrieved from STEP model libraries
and used in creating inspection process plans by applying
linear planning techniques. The inspection process planning
result was used by the inspection path planner to generate
collision-free probe paths that contained two portions: a
global path and a local path. The global path is for the
connecting relationship between different features of the
part under inspection; while the local path is the probe
trajectory for measuring the individual features.
The CMM path planning in Ref. [2] took the CAD model
and the generated sampling points as the input. Design
model entities were used to define the regions of the objects
to be measured and the measurement was performed for
each entity in turn. In some cases, two or more entities were
measured as a single surface. The measurements were
arranged in unidirectional or bi-directional scans depending
on the surface types. The programmed path could also be
edited by users.
5.2. Laser and optical measurement planning
As with the planning of CMM measurements, issues
exist for non-contact inspections such as laser scanning and
vision inspection. The visibility problem of vision inspection is very similar to the accessibility analysis for straight
probes [81]; and similarly, the accessibility in laser scanning
has an analogy with that of CMM-based inspection, or
machine vision-based inspection [50]. In sensor planning of
visual inspection, the objective is the automatic determination of the appropriate sensor settings when the requirements of the task are given [14]. Planning for laser scanning
inspection is the automatic generation of the number and
direction of scans, and of the scan paths.
In many of existing non-contact inspections including
vision and laser scanning, substantial human involvement
using laborious and time-consuming techniques is still
required [14,50]. Such processes are not cost effective and
lack flexibility. In recent years, some research has been
proposed to develop automated planning for visual or laser
inspections. Shen et al. [89] introduced a CAD-guided
camera positioning system to aid 3D part inspection by
structured light. The geometric information in the CAD
model and the camera model were used to plan camera
configurations that satisfied certain task constraints. Vision
sensor planning consisted of two main steps: find flat
patches and determine the viewpoints for each flat patch.
The flat patch was determined by any surface normal
vectors inside the patch and the patchs average surface
normal vector. The viewpoints were determined from a
recursive approach.
Three planning algorithms were proposed in Ref. [21] for

finding a set of sensing operations for the completely visual
measuring of the exposed surface of an object. The first
algorithm was for use in general cases. The second
algorithm was for the case when a plan was short and
generated quickly; while the third was for a longer plan and
higher data acquiring accuracy. According to the authors,
the above algorithms should be chosen based on application
requirements.
To find an optimal plan was intractable, so Lee et al. [50]
proposed algorithms for the automation of laser scanner
based inspection operations. The algorithms consisted of the
following operations: based on the constraints existing in a
laser scanning operation (such as satisfying the view angle,
the depth of view, checking interference with a part, and
avoiding collision with the probe), the generation of all
possible accessible directions at each sampled point on a
part surface; the calculation of the number of scans and the
most desired direction for each scan; and, the determination
of the scan path satisfying the minimum scan time.
Yang [14] presented an optimized sensor planning
system for active visual inspection of 3D models by
dimensioning a set of edge segments such that the active
vision errors were minimized and the sensor constraints
were satisfied. This research considered quantization errors
and displacement errors, which are considered inevitable in
active vision inspections. The combined density functions
of these errors are used as objective functions in the
optimizations. The constraints of resolution, focus, field-ofview, and visibility were used as the constraint functions.
The genetic algorithm was used for optimization so that the
resulting sensor setting would have the minimum active
vision errors and satisfy all sensor constraints. Given the
CAD model and the entities to be inspected, the active
visual inspection planning system was designed to obtain
the sensor setting that maximized the probabilities of a
required accuracy for each entity.
6. Inspection of free-form surface with datums

In manufacturing, tolerances are assigned to the
surfaces to control the quality of the free-form surfaces.
A tolerance is a specification that defines the acceptable
variation of dimension or geometry of an element. Two
types of tolerances are usually used: dimensional and
angular tolerances; and, geometric tolerances [70]. In
checking tolerance, some research used intensity images,
some used information from CMM. For parts with freeform surfaces and a relatively few flat surfaces, the
geometric tolerances specified for this type of surface, or
the normally assigned tolerances, are the profile tolerances
of a surface. Such tolerances are not easily inspected by
the traditional techniques of metrology [70,91]. The profile
tolerances assigned can be classified into individual and
related profile tolerances, based on whether it is designed
with a datum reference [91]. Research has been carried

out on the inspection of free-form surfaces assigned
tolerance with datum.
Ge et al. [20] proposed the tolerance specification
module and the comparative analysis module. The first
module was to specify tolerance information to the 3D CAD
model of a part. The comparative analysis module was to
construct datum reference frames and compare the
measurement data of an actual feature with its nominal
design, based on the tolerance requirements so that the
dimensional quality of the actual feature could be verified.
To construct the datum reference frames, a best fit method
along with a statistical analysis were used to find the best
fitted datum feature, which was built based on the
measurement data by a non-linear least-square minimization approach. The objective function was the sum of
squared normal deviations calculated between the measurement points and the fitted features in normal direction. A
number of non-linear equations needed to be solved to
determine the parameters of the datum features. All the
datum reference frames were constructed by using two
elementary datum features: plane and cylinders. In detailed
comparative analysis, individual features and independent
datum features were verified first, and dependent datum
features, which were related features or used as datum
features, were verified second. While those related features
that were not used as datums were verified lastly. Prieto et al
[70,71] defined and implemented a methodology to check
dimensional, angular, and geometric tolerances by using a
set of 3D point clouds and a CAD model of the part. The 3D
point clouds were registered with the CAD model and then
segmented into different surface patches by using the CAD
model. Finally, the specified tolerances were verified. In
detailed tolerance checking for free-form surface, the 3D
measurement points after segmentation were registered with
the CAD model, and the perpendicular distance between
each 3D point and the NURBS surfaces was calculated. The
distribution of these distances was used to define the zone of
measured tolerance. Limitations exist to above method
since the surface inspection in practical engineering
applications is much more complex. The free-form surface
tolerance may have a relationship with more than one
datum, and these datums also deviate from their nominal or
their design models. The localizations of free-form surface
and the design datums influence to each other. This problem
was addressed in Ref. [91]. Huang et al. [91] extended the
relative algorithms on inspection of individual free-form
surface to cases of sculptured surfaces with design datums.
According to the authors, the inspection strategy for
tolerances with datums was different from that used in
individual tolerance inspection. There were two steps for
this case. The first step was the localization of the design
frame by using datum reference features instead of the freeform surface itself. The second step was the re-localization
of the free-form surface within allowable ranges inside the
datum tolerance zone to minimize the deviations. In this
1413
second, the localization results from the datum reference

frame were used as the initial values to check if the changes
are out of the tolerance of datums. In localizing the DCS
with datum frames, the research used the least-square
method and represented the homogeneous transformation
matrix Td by
p
Td rUp rUT r 0U rUT 21
15
where rUp represents a set of measurement points, and rU

represents the corresponding point set from design model.
The localization process was carried out from the datum
features one by one. The detailed process is shown in
Fig. 11. The deviation of a manufactured surface was
represented in two components: deterministic error and
random error. It was assumed that the random error was the
dominant error; and, the tolerance verification was carried
out based on statistical analysis. The shortcoming of this
approach is that corresponding points from those design
datums have to be decided in order to solve the
transformation information in the first step localization.
This makes the process a little bit complicated.
7. Brief description of sample commercial inspection

packages
There are a number of commercial packages available for
inspection and comparison purposes. Some of these systems
are based on contact measurements such as CMM
measurement; the others are based on non-contact measurements such as optical or laser scanners, or on both contact
and non-contact measurements. Most of these packages can
handle free-form surfaces. However, in many of the
systems, user interaction may be required. For example,
the two surfaces under inspection have to be set close
enough by manual operation. Earlier visual inspection
packages were reviewed in Ref. [85], such as the automobile
body dimensional inspection systems developed by General
Motors and Volkswagen, and the turbine blade inspection
system developed by Westinghouse. Some examples of the
existing commercial packages are discussed in the following paragraphs.
A geometric inspection system for the comparison of
measured point sets with geometric models was introduced
in Ref. [31]. There were two stages in the inspection and
comparison process. In the first stage, an operator manually
moved geometric models to the measurement points and 3D
transformations were also made manually. In the second
stage, fine alignment was implemented to minimize the
deviation function. The system used the STL data format for
describing the geometric model. This was a stand-alone
system.
Ref. [32] reported a 3D optical scanning and inspection
system for measuring the shape and wear-out amount of
tires. Ref. [33] showed a product called SPV, namely
1414
Fig. 11. Calculation procedure for localization with datum [91].
Sitius Part Verification. The measurement points can be

obtained through CMM, laser scanners or optical scanners.
SPV was fully integrated into Unigraphics and CATIA
platforms. The package can be applied for the inspection of
many mechanical products including free-form shaped
parts.
The functions for product verification and inspection was
described in Ref. [34]. The first function was the comparison
of scanned data from the physical part with the nominal
CAD model. The second function was the comparison
between two sets of scanned point clouds, or between a
dense point cloud and the CAD model. This inspection
service has some applications in automotive industry, such
as product part approval prior to first product shipment at
Chrysler, Ford and GM. In Ref. [35], the inspection system
acquired a dense point cloud from products. Then the point
cloud and the CAD surface geometry were registered with
or without the use of datum. Deviations of the point cloud
from the CAD model were identified.
Ref. [36] used a product function called Computer
Aided Inspection/Verification to scan prototypes or
production components with a laser scanner and to compare

the scanning results directly with the CAD model for
verification. The system generated display plots showing
color gradients corresponding to point deviation from the
nominal CAD surface.
With the requirements and developments of reverse
engineering and its applications, a number of packages have
been developed, which, in addition to functions for creating
and manipulating reverse engineering models based on
scanned point data, have the functions of inspection and
comparison.
Some of the popular brand names on the market include:
Polyworks, RapidForm, Geomagic, Imageware, Metris and
SpatialAnalyzer [37 42]. These systems have similar
inspection and comparison procedures that input the design
model and measurement data, then align them into a
common coordinate system, and finally make comparisons
and output the inspection results. The methods for the
alignment between design model and measurement data by
these systems normally include a traditional 3-2-1 approach,
semi-automatic and automatic processes such as best fit
1415
and feature-based alignment. With the semi-automatic

processes, users need to make the initial alignment by
manually arranging the design model and measurement data
close enough. The systems, then, carry out the remainder of
the registration operations. Features are used for the
automatic alignment including planes, circles, lines, spheres
and/or some other quadratic surfaces. The inspection and
comparison results are outputted in a report and displayed
visually by color gradients.
improvements of efficient and effective inspections for

higher quality product design and manufacturing.
8. Discussions and conclusions
References
There existed a number of literature reviews on visual

inspection and automated visual inspection techniques
based on the literature published prior to 1995. None of
the published reviews were concerned with the contact
inspection approach such as CMM-based inspection, which
is still widely used in industrial inspection processes. This
paper provided a review of the existing free-form surface
inspection techniques. From a higher level viewpoint, this
paper covered both contact and non-contact measurement
based free-form surface inspection approaches. In this
review contact inspection was mainly focused on CMM
measurement, while non-contact inspection was based on
laser and optical scanning. Characteristics, advantages and
disadvantages of the inspection approaches were discussed.
This paper also reviewed all of the major processes and
techniques involved in free-form surface inspection. These
included: the measurement data acquisition methods by
both contact and non-contact measurements; the measured
data and/or design model geometric modeling techniques;
and, the surface localization and registration methodologies
including the establishment of correspondence and 3D
transformation, and the comparison between the measurement data and the design surface. Various practical issues
such as the measurement point selection, factors influencing
localization process, inspection planning, tolerance inspection and inspection with design datum were discussed. Some
existing commercial inspection packages were also briefly
reviewed.
Future research on free-form surface inspection will
likely focus on the development of inspection techniques
with higher accuracy, efficiency and robustness, and
reduced cost. The measurement efficiency of contact
measurement devices such as CMM will probably improve,
while still maintaining the high accuracy, by combining
CMM with non-contact sensors. The accuracy of noncontact measurement approach is expected to be enhanced
by using higher accuracy sensors and optimizing the
measuring parameters. Increased speed and higher accuracy
of inspections should be fully explored. New algorithms are
also expected to be developed for the more robust and
efficient processing of various types of measurement data.
The decreased cost of these solutions is also an important issue. These are all essential factors for future
[1] Marshall AD, Martin RR. Automatic inspection of mechanical parts

using geometric models and laser range finder data. Image Vis
Comput 1991;9(6):385405.
[2] Ainsworth I, Ristic M, Brujic D, Visual CAD. Visual CAD based
measurement and path planning for free form shapes. Seventh Int
Conf Central Europe Comput Graphics, Visual Interact Digital
Media99, Prague 99 A.D; 2000.
[3] Spyridi AJ. Automatic generation of high level inspection plans for
coordinate measuring machines. PhD Dissertation. University of
Southern California; 1994
[4] Balasubramanian S, Gu P. A neural network approach to localization
of parts with complex surfaces for precision inspection. Proc ASME
Comput Engng Conf 1995;1995.
[5] Horn BKP. Closed form solution of absolute orientation using unit
quaternions. J Optical Soc A 1987;4(4):629 42.
[6] Horn BKP, Hilden HM, Negahdaripour S. Closed form solution of
absolute orientation using orthonormal matrices. J Optical Soc A
1988;5(7):112735.
[7] http://www.brownandsharpe.com/introduction/
[8] http://www.brownandsharpe.com/mfg/mfg5/index.html
[9] Brenner C, Bohm J, Guhring J. Experimental measurement system for
industrial inspection of 3D parts. Proc of SPIE Vol. 3521, Techn Conf
3521 (352127); 1998.
[10] Dorai C, Weng J, Jain AK. Optimal registration of multiple range
views. Technical Report CPS 1994;94-36:11.
[11] Unsalan C. Aytul ERCIL, Automated tolerance inspection by implicit
polynomials. Proc ICIAP99 Venice, Italy, Septmber; 1999. [pp.
1218 25].
[12] Chua CS, Jarvis R. 3-D free-form surface registration and object
recognition. Int J Comput Vis 1996;17:7799.
[13] Menq CH, Yau HT, Lai GY. Automated precision measurement of
surface profile in CAD-directed inspection. IEEE Trans Robotics
Automation 1992;8(2):26878.
[14] Yang CC, Ciarallo FW. Optimized sensor placement for active visual
inspection. J Robotic Syst 2001;18(1):1 15.
[15] Zhang D. Harmonic shape image: a 3D free-form surface representation and its application in surface matching. Doctoral Dissertation,
Robotics Institute, Carnegie Mellon University; 1999
[16] Bispo EM, Fisher RB. Free-form surface matching for surface
inspection. IMA Conf Mathematics Surfaces; 1994. [pp. 11936].
[17] Faugeras OD, Herbert M. The representation recognition and
localization of 3-D objects. Int J Robotic Res 1986;5(3):27 52.
[18] Blais G, Levine D. Registering multi-view range data to create 3D
computer objects. IEEE Trans PAMI 1995;17:820 4.
[19] Gregory C. ICP registration using invariant features. IEEE Trans
Pattern Anal Machine Intell, January 2002;24(1):90102.
[20] Ge Q, Chen B, Smith B, Menq CH. Tolerance specification and
comparative analysis for computer-integrated dimensional inspection.
Int J Prod Res 1992;30(9):217397.
[21] Tarbox GH, Gottschlich SN. Planning for complete sensor coverage in
inspection. Comput Vis Image Understanding 1995;61(1):84 111.
Acknowledgements
The authors wish to thank the Natural Sciences and
Engineering Research Council of Canada (NSERC) for
providing this research with financial support.
1416
[22] Grimson WEL, Lozano-Perez T. Localizing overlapping parts by

searching an interpreting tree. IEEE Trans PAMI 1987;9(4):469 82.
[23] Turk G, Levoy M. Zipped polygon meshes from range images.
Comput Graphics (SIGGRAPH94 Proceedings); 1994. [pp. 311 18].
[24] Gunnarson KT, Prinz FB. CAD model based localization of parts in
manufacturing. Computer 1987;20(8):6674.
[25] Delingette H, Seguin O, Perrocheau R, Menegazzi P. Accuracy
evaluation of 3D reconstruction from ct-scan images for inspection of
industrial parts. In: Cepadues, editor. Proc Conf Quality Control
Artificial Vison (QCAV97), Le Creusot, France, May. 1997.
[26] Pahk HJ, Ahn WJ. Precision inspection system for aircraft parts
having very thin features based on CAD/CAI integration. Int J Adv
Manufact Technol 1996;12:4429.
[27] Pottmann H, Wallner J, Leopoldseder S. Kinematical methods for the
classification, reconstruction and inspection of surfaces. In: Proceedings SMAI, Pompadour, France; 2001.
[28] Yau H, Menq C. Automated CMM path planning for dimensional
inspection of dies and molds having complex surfaces. Int J Machine
Tools Manufact 1995;35(6):86176.
[29] Huang X, Xiong Y. Profile tolerance evaluation of parametric curves
and surfaces. Chinese J Mech Engng 2002;15(2):1215.
[30] Ainsworth I, Ristic M, Brujic D. CAD-Based Measurement Path
Planning for Free-form Shapes Using Contact Probes. Int J Adv
Manufact Technol 2000;16:2331.
[31] http://www.aha.ru/labsapr/geocon.htm
[32] http://www.oes.itri.org.tw/s200/3d.htm ^ fitting ^ of ^ free-form
[33] http://www.sitius.com
[34] http://www.gks3d.com/service.htm
[35] http://www.firstart.com/inspect_procedure.html
[36] http://www.laserdesign.com/cav.htm
[37] http://www.innovmetric.com
[38] http://www.rapidform.com
[39] http://www.geomagic.com/advantage/testimonials/
[40] http://www.eds.com
[41] http://www.metris.be/
[42] http://www.kinematics.com
[43] http://www.hymarc.com/index.html
[44] http://shapegrabber.com/
[45] http://www.minolta.com
[46] http://www.metronsys.com
[47] Guehing J. Reliable 3D surface acquisition, registration and validation
using statistical error models. Third Int Conf 3-D Digital Imaging
Modeling (3DIM01), Quebec City, Canada, May 28June1; 2001.
[48] Fan KC, Tsai TH. Optimal shape error analysis of the matching image
for a free-form surface. Robotics Comput Integ Manufact 2001;17:
21522.
[49] Sahoo KC, Menq CH. Localization of 3-D objects having complex
sculptured surfaces using tactile sensing and surface description.
J Engng Indus 1991;113:8592.
[50] Lee KH, Park HP. Automated inspection planning of free-form shape
parts by laser scanning. Robotics Comput-Integ Manufact 2000;16(4).
[51] Kanatani K. Analysis of 3-D rotation fitting. IEEE Trans Pattern Anal
Machine Intell 1994;16(5).
[52] Kase K, Makinouchi A, Nakagawa T, Suzuki H, Kimura F. Shape
error evaluation method of free-form surfaces. Computer-Aided Des
1999;31:495 505.
[53] Arun KS, Huang TS, Blostein SD. Least-squares fitting of two 3-D
point sets. IEEE Trans Pattern Anal Machine Intell 1987;9(5).
[54] Bardis L, Jinkerson RA, Patrikalakis NM. Localization for automated
inspection of curved surfaces. Int J Offshore Polar Engng 1991;1(3):
22834.
[55] Chen LC. Surface digitization techniques for reverse engineering,
http://www.cc.ntut.edu.tw/coop/atc/aoi/html2/tecedu.html, 2002
[56] Li L, Schemenauer N, Peng X, Zeng Y, Gu P. Design and
implementation of a reverse engineering system for rapid manufacturing 2002;18(1):5367.
[57] Lorusso A, Eggert D, Fisher RB. A comparison of four algorithms for

estimating 3-D rigid transformations. Proc British Machine Vis Conf
BMVC95; 1995. [pp. 237 46].
[58] Bariani M, Cucchiara R, Mello P, Piccardi M. Data mining for
automated visual inspection, in Proc. of PADD97. First Int Conf
Practical Appl Knowledge Discovery and Data Mining, London, UK;
1997. [(ISBN 0-9525554-7-6), pp. 51 64].
[59] Menq CH, Yau HT. Path planning for automated dimensional
inspection using coordinate measuring machines. Proc IEEE Int
Conf Robotics Automation, April 1991;3:19349.
[60] Nashman M, Hong TH, Rippey WG, Herman M. An integrated vision
touch-probe system for dimensional inspection tasks. Proc Int
Conf Intell Syst: a Semiotic Perspective, Gaithersburg, MD; 1996.
[pp. 196 205].
[61] http://www.olympus.net/personal/mortenson/preview/definitionsf/
freeformsurface.html
[62] Siegel M, Gunatilake P. Remote inspection technologies for aircraft
skin inspection. Proc ET VS-IM, IEEE; 1997. [pp. 6978].
[63] Patrikalakis NM, Bardis L. Localization of rational B-spline surfaces.
Engng Comput 1991;7:23752.
[64] Ohta N, Kanatant K. Optimal estimation of three-dimensional rotation
and reliability evaluation. IEICE Trans Inf Syst 1998;E82(D(11)).
[65] Boulanger P, Moron V, Redarce T. High-speed and non-contact
validation of rapid prototyping parts. SPIEs Rapid Product
Development Technologies Boston, Mass. USA, Nov.1819; 1996.
[pp. 46 60].
[66] Gu P, Chan K. Generative inspection process and probe path planning
for coordinate measuring machines. J Manufact Syst 1996;15(4):
240 255.
[67] Gunatilake P, Siegel M, Jordan A, Podnar G. Image understanding
algorithms for remote visual inspection of aircraft surfaces. Machine
Vis Appl Indus Inspection V 1997;3029:213.
[68] Besl PJ, McKay ND. A method for registration of 3-D shapes. IEEE
Trans Pattern Anal Machine Intell 1992;14(2):23956.
[69] Besl PJ. The free-form surface matching problem. Machine Vision for
Three-Dimensional Scenes. H. Freeman (Ed.), Academic Press Inc.,
CA, 1990. [pp. 25 71].
[70] Prieto F, Redarce T, Lepage R, Boulanger P. An automated inspection
system. Int J Adv Manufact Technol 2002;19:91725.
[71] Prieto F, Lepage R, Boulanger P, Redarce T. Inspection of 3D parts
using high accuracy range data. SPIE Conf Electron Imaging:
Machine Vis Appl Indus Inspect VIII, (EI SPIE, San Jose, USA,
San Jose, USA, 23 28 January 2000, vol. 3966.; 2000. [pp. 8293].
[72] Prieto F, Redarce T, Lepage R, Boulanger P. A non contact CADbased inspection system. Int Conf Quality Control Artif Vis
(QCAV99), Canada: Trois-Rivie`res; 1999. [pp. 133 38].
[73] Prieto F, Redarce HT, Lepage R, Boulanger P. Visual system for the
fast and automated inspection of 3D parts. Proc 6ie` me
Assises Europeennes de Prototypage Rapide Paris, France; 1998.
[pp. 211 27].
[74] Jinkerson RA, Abrams SL, Bardis L, Chryssostomidis C, Clement A,
Patrikaiskis NM, Wolter FE. Inspection and feature extraction of
marine propellers. J Ship Prod 1993;9(2):88106.
[75] Langenbach R, Ohl A, Scharf P, Smmler J. Automatic visual
inspection system for small stampings with free-form surfaces. Proc
SPIE 2001;4301:919.
[76] Campbell RJ, Flynn PJ. A survey of free-form object representation
and recognition techniques. Comput Vis Image Understand 2001;81:
166 210.
[77] Rich Pito, Bajcsy, R., A solution to the next best view problem for
automated CAD model acquisition of free-form objects using range
cameras, in the modelling, simulation, and control technologies for
manufacturing session of the SPIE Symposium on Intelligent Systems
and Advanced Manufacturing, Philadelphia PA, October; 1995
[78] Chin RT, Harlow CA. Automated visual inspection: a survey. IEEE
Trans Pattern Anal Machine Intell 1982;6:55773.

[79] Chin RT. Automated visual inspection: 1981 to 1987. Comput Vis,
Graphics Image Process 1988;41:346 81.
[80] Volpe R, Balaram J. Technology for robotic surface inspection in
space. Proc AIAA/NASA Conf Intell Tobots Field, Factory, Service
Space 1994;(CIRFFSS94):214.
[81] Spitz SN. Dimensional inspection planning for coordinate measuring
machines, PhD Dissertation, University of Southern California; 1999
[82] Masuda T, Yokoya N. A robust method for registration and
segmentation of multiple range images. Comput Vis Image Understand 1995;61(3):295 307.
[83] Shen T, Huang J, Menq CH. Multiple-sensor integration for rapid and
high-precision coordinate metrology. IEEE/ASME Trans Mechatron
2000;5(2):11021.
[84] Newman TS, Jain AK. A system for 3D CAD-based inspection using
range images. Pattern Recogn 1995;28(10):155574.
[85] Newman TS, Jain AK. A survey of automated visual inspection.
Comput Vis Image Understand 1995;6(2):23162.
[86] Umeyama S. Least-squares estimation of transformation parameters
between two point patterns. IEEE Trans Pattern Anal Machine Intell
1991;13(4):37680.
[87] Walker MW, Shao L, Volz RA. Estimating 3-D location parameters
using dual number quaternions. CVGIP: Image Understand 1991;
54(3):35867.
[88] Cheng WL, Menq CH. Integrated laser/CMM system for the
dimensional inspection of objects made of soft material. Int J Adv
Manufact Technol 1995;10:36 45.
[89] Sheng W, Xi N, Song M, Chen Y, Rankin JS. Automated CAD-guided
automobile part dimensional inspection. Proc IEEE Int Conf Robotics
Automation; 2000. [pp.115762].
[90] Wolovich W, Albakri H, Yalcin H. The precise measurement of freeform surfaces. J Manufact Sci Engng 2002;124(1).
[91] Huang X, Gu P. CAD-model based inspection of sculptured surfaces
with datums. Int J Prod Res 1998;36(5):1351 67.
1417
[92] Huang X, Gu P, Zernicke R. Localization and comparison of two freeform surfaces. Comput-Aided Des 1996;28(12):101722.
[93] Huang X, Gu P. CAD-model based inspection of sculptured surfaces.
Proc ASME Comput Engng Conf; 1995. [pp. 795804].
[94] Chen Y, Medioni G. Object modeling by registration of multiple range
images. Image Vis Comput 1992;10:14555.
[95] Li Y, Gu P. Geometric estimation of 3D machined parts for internetbased manufacturing system. Proc CIRP Conf Workshop, Hong
Kong, May; 2002.
[96] Li Z, Gou J, Chu Y. Geometric algorithms for workpiece localization.
IEEE Trans Robotics Automation 1998;14(6):86478.
Peihua Gu is currently Professor and Head of

Mechanical and Manufacturing Engineering
Department at the University of Calgary. He
also holds NSERC Chair in Life Cycle Design
Engineering. His research interests include life
cycle design, rapid product realization and
inspection and assembly.
Yadong Li is a PhD candidate at The
University of Calgary. His research interests
include CAD/CAM, Geometric Modeling,
Inspection of Free-form Surface and CNC
Programming/Machining Technology. He
received his BS and MS degrees from Tianjin
University. He had worked as mechanical
engineer on die/mold manufacturing in TONTEC Co., research assistant in The Chinese
University of Hong Kong and mechanical engineer in R&D department of
Makino Milling Machine Co., Ltd.

2X - Free-Form Surface Inspection Techniques State of The Art Review

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2X - Free-Form Surface Inspection Techniques State of The Art Review

Uploaded by

Copyright:

Available Formats

Computer-Aided Design 36 (2004) 13951417

Free-form surface inspection techniques state of the art review

the inspection techniques and equipment are mature in

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

facilities: contact measurements, normally by using a

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

some commercial inspection packages. Finally, conclusions

2. Measurement data acquisition methods

Fig. 2. Measurement Modeling [91].

a contact probe. In this figure, Pi is a design model point

Traditional inspection of free-form surfaces has involved

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

Fig. 3. Classification of various digitizing devices [55].

a slower measuring speed than laser and optical scanners.

fixture and the needs of programming when a different part is

Fig. 4. CMM measurement of a part [7].

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

Fig. 5. Non-contact measurement illustration [7].

the tactile probe will have problems accessing part features.

systems can be as good as about 0.00100 (0.025 mm) [43,70,

Fig. 6. Three types of standard optical technology [7].

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

sensor replaced the probe so as to eliminate the possible

3. Surface description methods for localization

the measurement surface was described by a general

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

presented a method for evaluating the profile tolerance of

In comparison, the parametric representations such as

The fitting process was based on so called 3L fitting

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

fixtures or other part presentation/orientation devices totally

Fig. 7. Traditional 3 21 approach for localization ([13]).

no accurate result can be uniquely decided by choosing

where d 2 rij is the minimum distance between an arbitrary

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

smaller RMS value after transformation compared with the

arbitrary measurement point rip and its closest point (defined

For a given set of point rUp representing measured surface

T 21 rU rUT rUp rUT 21

The following illustrates the procedures of the localization

where T is the rigid transformation matrix to be determined,

Step 1: Using initial rU from the designed surface model

The above algorithm was extended to localization with

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

of those interested regions was required at the initial state.

The automated visual inspection system introduced in

4.2. Localization techniques in approaches handling

Fig. 8. Block diagram of the inspection system [65].

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

the alignment between the measurement object and the

where d was a point from the design model; s was the

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

as a least-square minimization problem. If the estimation of

bi-cubic splines surface patch, Gunnarson and Prinz [24]

where xmi ; ymi ; zmi is an arbitrary measurement point,

The parameters u;v can be found to determine the

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

where u and v were design surface parameters; T was

where k01 and k02 are the principle curvatures at the

Fig. 9. Point to NURBS surface distance [65].

In Ref. [48], the nearest distance between two sets of

Y. Li, P. Gu / Computer-Aided Design 36 (2004) 13951417

4.4. Measurement point selection, accuracy, efficiency