IEEE TRANSACTIONS ON PATTERN ANALYSIS A N D MACHINE INTELLIGENCE. VOL. 11.

NO

I?.

DECEMBER 198)

1293

Subpixel Measurements Using a Moment-Based Edge Operator

EDWARD P. LYVERS, OWEN ROBERT MITCHELL, SENIOR MEMBER, IEEE, MARK L. AKEY, MEMBER, IEEE, AND ANTHONY P. REEVES, SENIOR MEMBER, IEEE

Abstract-This paper presents recent results in precision measurements using computer vision. An edge operator based on two-dimensional spatial moments is presented. The operator can be implemented for virtually any size window and has been shown to locate edges in digitized images to a twentieth of a pixel. This accuracy is unaffected by additive o r multiplicative changes to the data values. The precision is achieved by correcting for many of the deterministic errors caused by nonideal edge profiles using a look-up table to correct the original estimates of edge orientation and location. This table is generated using a synthesized edge which is located a t various subpixel locations and various orientations. The operator is extended to accommodate nonideal edge profiles and rectangularly sampled pixels. The application of this technique to the measurement of imaged machined metal parts is also presented. Additionally, both theortical and experimental noise analyses show the operator has a relatively small bias in the presence of noise.
Index Terms-Edge detection, edge location, facet model, Hueckel, image processing, Laplacian of a Gaussian, spatial moments, subpixel measurement.

I. INTRODUCTION NTIL recently, the location of features in digital imagery to within a pixel has seemed adequate. For those cases which require higher precision, the primary solution has been to increase the sampling rate. However, there are applications in the digital image processing field that need better accuracy. These include photogrammetry and industrial inspection, where measurement accuracy is at a premium. Up to this point few edge locators have been capable of subpixel accuracy. Frei and Chen [l], Roberts [2], and Sobel [3] have all proposed edge detection schemes which use some digital approximation to the gradient. These methods are very fast and deliver acceptable results; however, the locations are not subpixel and the derivative operation is very sensitive to noise. Macvicar-Whelan and Binford [4] have proposed a method which utilizes the gradient operation after the data has been smoothed within an arbitrary odd-sized window. The subpixel edge loca-

Manuscript received September 30, 1984; revised July 20, 1989. Recommended for acceptance by W. E. L. Grimson. E. P. Lyvers is with MIT Lincoln Laboratory, Lexington, MA 01273. 0. R. Mitchell is with the Department of Electrical Engineering, University of Texas at Arlington, Arlington, TX 76019. M. L. Akey is with Magnavox Electronics System Company, Fort Wayne, IN 46808. A. P. Reeves is with the School of Electrical Engineering, Cornel1 University, Ithaca, NY 14853. IEEE Log Number 893 1117.

tions are determined by linearly interpolating the location within a rise-fall-rise region of the gradient. This method is much less sensitive to noise in comparison to the other gradient operators; however, precision is limited to interpolation between a few points. Machuca and Gilbert [5] have proposed a method which integrates the region containing the edge. This method uses the moments found in the region to determine the position of the edge. Nevatia and Babu [6] have designed a linear feature extractor using edge detection and line thinning. Their edge detector convolves windowed data with six edge masks. The maximum response from this matched filter is chosen as the correct edge orientation of an edge centered in the window. Hueckel [7]-[9] has developed an algorithm to determine the presence of edges and lines by fitting a region of data to a Hilbert space with 9 parameters. The locations are to subpixel accuracies; however, no analysis of the operator in the presence of noise is documented. Tabatabai and Mitchell [lo] have developed an operator which locates edges by fitting the first three gray level moments to the data. Again, the edge locations are to subpixel accuracies as well as being much less sensitive to noise when compared to the Hueckel operator. Huertas and Medioni [ l 11 have developed a subpixel operator using the facet model combined with Laplacian of Gaussian masks. Englander [12] describes a subpixel operator based on image resampling using Whittakers theorem. This paper presents a new edge operator (detector) which uses the spatial moments of a gray level edge to determine the location of the edge. Section I1 defines the one-dimensional edge operator. The operator is evaluated for subpixel accuracy as well as performance in the presence of noise. The Tabatabai operator is implemented in one dimension and compared to the moment based edge operator. Section I11 defines the two-dimensional edge operator. Again, the operator is tested for subpixel accuracy in addition to performance in the presence of noise. The Tabatabai, the Hueckel, and a modified version of the Huertas and Medioni edge operators are compared to the moment based edge operator. A confidence measure is developed for the two-dimensional operator. In addition, recent tests of the method on precision edge contrast and orientation estimation [ 131 show this new moment method to outperform Sobel, Nevatia-Babu, facet model, and Gaussian smoothed gradient techniques.

0162-8828/89/1200-1293$01.OO 0 1989 IEEE

1294

IEEE TRANSACTIONS ON PATTERN ANALYSIS A N D MACHINE INTELLIGENCE. VOL. 1 1 . NO. I ? . DECEMBER 1989

B. Bias Efects Due to Pixel Sampling In the previous section, the moments of an ideal edge in a continuous domain were solved for the parameters of that edge ( h , k, 1 ) . However, the edge model does not allow for the sampling effects due to finite pixel width in real data; i.e., the gray value is assumed constant over A . Derivation of the One-Dimensional Edge Operator An ideal one-dimensional edge model is shown in Fig. each pixel in real data. This results in a bias error in the 1. The model is characterized by three parameters: back- calculated edge location. Fig. 2 shows this bias error beground intensity h, edge contrast k, and edge translation tween the calculated edge location and the actual edge 1. The edge is simply the step transition from gray level location. The sampled edge is generated from an ideal h to gray level h + k. The edge translation 1 is defined to continuous edge that is sampled with a square aperture of be the length from the center of the edge model to the step width one pixel. The plot shows the bias error when the ideal edge location is vaned from - 1.5 to 1.5 pixels from transition and is confined to the range of - 1 to + 1. the center of a window of width 5 pixels. The moments of a continuous functionf(x) of order p For comparison, the edge operator developed by Taare defined by batabai and Mitchell [lo]was implemented. It, too, has the ability to locate edges to subpixel accuracy. It should Mp = x p f ( x ) ah. (1) be noted that the edge operator developed in this paper Since the model edge is completely characterized by three generates moments using spatial information. The posiparameters, three suitable moments of the edge in terms tion of the pixel is used in the moment kernel functions of h, k, and 1 are required to solve for each parameter and the gray value of the pixel simply weights those funcsimultaneously. The desired moments can be found using tions. On the other hand, the Tabatabai operator generates ( l ) , where f (x) is the edge function. moments from the gray value only. No spatial information is used in determining these moments. To avoid conM = h i l l a h + k ! a h = 2h + k(l - I ) O ( 2 ) fusion when refemng to these two operators, the moment1 based operator will be called the spatial moment edge op1 1 erator. The Tabatabai operator will be called the gray level MI = h xdx + k ah = ik(l - 1) ( 3 ) moment edge operator. The bias error for the gray level -1 I moment operator is also shown in Fig. 2 . For both operators, when the edge location perfectly M2 = h x 2 a h + k 1 x 2 a h = j h + ik(1 - 1 3 ) matches pixel boundaries the error is zero. In these in-I I stances, the sampled edge is identical to the continuous ( 4 ) edge. However, when the edge lies between pixel boundThe equations (2), ( 3 ) , and (4) may now be combined to aries, the sampled edge actually contains three distinct gray values rather than two. The intermediate gray value solve for 1. is determined by the position of the continuous edge. The ( 5 ) moments generated from these two edge types differ. Thus, the length calculation produces an error in the acThe parameter k may be obtained by substituting the value tual length to the edge. To further understand these differences, for the spatial of 1 back into ( 3 ) , and h obtained from ( 2 ) using the values moment operator, consider the case of the sampled edge k and 1. in Fig. 3 . Note that the variables lI and l2 are measured to pixel boundaries and thus are discrete functions of I with l1 I1 I1 2 . The moments MO,MI, and M2 are calculated to be

11. ONE-DIMENSIONAL OPERATOR EDGE The new edge operator uses the spatial moments of the discrete edge data to determine the position of an edge. The moments of an ideal edge in a continuous domain are determined and the position of the edge is mathematically derived from these moments. The position results are then modified to correct for deterministic errors due to the sampling. The edge operator is then applied to noisy data. Theoretical and empirical statistics are determined. Lastly, a one-dimensional gray level moment edge operator developed by Tabatabai and Mitchell [lo]is implemented and its performance is compared to the spatial moment edge operator results.

h+k
I
1

-1

Fig. 1, Ideal continuous, one-dimensional, edge model characterized by h , k , and 1.

1 1

MO= jllhak+
Thus, sample moments can be used to estimate M O , M I , and M 2 , and then ( 5 ) used to estimate the edge location.
=

j Akdx+
12

II

jIZ k d x
- Z2)

2h

+ Ak(Z2 - 11) + k( 1

(8)

LYVERS er al.: SUBPIXEL MEASUREMENTS USING A N EDGE OPERATOR

1295

per step 1,. The sampling aperture is flat with a width of one pixel. The constant ( l2 - l1) is the length of one pixel. Solving (12) for 1 gives
1=
12
- -

Ak (& k

11).

(13)

The bias error is simply the actual length 1 minus the moment calculated length l,,.,,
B ( l l , 1 2 , Ak, k ) = 1 1M.

(14)

-1.5

-1.0

-.50

00 .

.50

1.0

1.5

Substituting (1 1) and (12) into the bias equation and simplifying yields

Edge Translation,pixels

Fig. 2 . Edge translation error. The spatial moment operator is shown as a solid line while the gray level moment operator is shown as a dashed line.

where
h+k

p=-

Ak k '

o r p 5 1 .

(16)

The bias error is zero for all roots of the numerator of ( 1 3 , i.e.,
-1

Fig. 3. Sampled ideal edge characterized by h, A k , k, l , , and 12.

P(1,

- 11)2(Z,

12)(P -

1)

0.

(17)

M~ =
=

hx2 dr

-1

A,

Akx'dr

J,,kx2
(10)

$ h + fAk(1; - I:)

+ f k ( 1 - 1;).

Substitution of (8), (9), and (IO) into the length solution, ( 5 ) , gives

I,,.,

3M2 - M O 2Ml Ak[ZI(l

1:)

12(1 - l ; ) ]

+ k12(1 - 1 : )
(11)

Ak(1: - 1 : )

+ k( 1 - 1 ; )

Equation (11) is the length result when the operator is applied to sampled edges. To determine the bias associated with the length given in (11) and the ideal length, an image formation model is needed. Let the ideal sampled edge be generated by the linear equation

which shows the incremental step gray value Ak is proportional to the normalized distance that 1 is from the up-

When Ak = 0 (0 = 0 ) or Ak = k (0 = 1 ) the length equations agree and the bias error is zero. This condition occurs when the edge coincides with a pixel boundary and the three-level edge is reduced to a two-level edge. At every pixel boundary, the bias error is zero. The other situation where the length equations match is when l1 = -12. This occurs when the length is near the center of the window. It should be noted that the difference l2 - Zl remains constant and is equal to one pixel width. For a window of 5 pixels, the lengths lI and 1, equal -0.2 and 0.2, respectively, when the edge is located within the center pixel of the window. All edge positions within this range produce no bias error from the length equation, as can be seen in Fig. 2. The location error is well behaved and it was empirically verified that the calculated edge location versus true edge location is a monotonic function. Therefore, to eliminate the bias error that is present when the edge location is not within the center pixel and not centered on pixel boundaries, a look-up table procedure can be used. In this particular implementation, the bias table was created from edge locations spaced at 0.05 pixel intervals. Linear interpolation was used to determine the bias between the intervals. Use of the bias table to correct the calculated length resulted in a reduction of the maximum error over the -1.5 to +1.5 pixel range from 0.101 to 0.0016 pixels. Note that although this result is for a range of - 1.5 IZ I1.5 pixels, the most critical range is -0.5 I1 I 0.5 pixels because for I 1 I > 0.5 pixels the edge is closer to a window centered on an adjacent pixel. Note that for 1 1I I0.5 pixels, the spatial moment operator has no error, and therefore a bias correction table is not necessary.

1296

JEEE TRANSACTIONS ON PATTERN ANALYSIS A N D M A C H l N E INTELLIGENCE. VOL. I I . NO. 12. DECEMBER 1989

Other effects may also be incorporated into the error bias table. For instance, the sampling effects caused by a Gaussian-type aperture, rather than the square aperture used here, could be accounted for in the bias table. Nonsymmetric edges which are caused by cylindrical surfaces can also be located precisely with the bias table. However, this requires the nonsymmetric edge to be well modeled so that the appropriate bias table may be determined. Three-Level Model: Some thought has been directed toward establishing the three gray level edge as an edge model and finding the solution for the length using the new models moments. In this implementation, the length solution would not be in error. However, due to the increased complexity of the edge model, five parameters are needed to describe it sufficiently. The five parameters are h , Ak, k , 1 1 , and 1,. If the operator is applied only to edges blurred over one pixel, l 2 - II will be known. Therefore, the first four moments of the edge need to be calculated and solved for h , A k , k , I , , and 12. The solution, however, is not readily apparent. The additional length variable increases the complexity, prohibiting a closed form solution. For this reason, the three level edge model is dropped in favor of the original edge model. The location error for the gray level operator was also well behaved and a look-up table procedure was used for it too. The bias table was created from edge locations spaced at 0.05 pixel intervals. Use of the bias table to correct the calculated length resulted in a reduction of the maximum error over the - 1.5 to + 1.5 pixel range from 0.136 to 0.045 pixels.

are given by mp =
= =

i=I K

c c,(i)[f(i)
+ rip.

+ rqi)]
K

C r=I
M,

c,(i>f(i> +

C i= I

c,(i) A ( i ) (22)

The random moment, therefore, can be viewed as a deterministic part plus a random part. Furthermore, li, is simply the weighted sum of K independent, zero-mean Gaussian random variables. The resulting random variable ti,, is Gaussian distributed and is well documented in the literature [ 141. In the case where K is of moderate value, the distribution of each A ( i ) need not even be Gaussian, but only reasonably concentrated about the mean. This condition stems from the well known central-limit theorem. The mean of the resulting Gaussian density is zero and the variance is the sum of variances from each random variable. Each individual variance is the weighted variance of the additive noise, i.e.,
a i p = [c;(l)

+ c;(K)]a2

(23)

where is2is the variance of the additive noise. For a window size of 5 pixels the numerator and denominator standard deviations are
oN = 0.71840

and aD = 1.01190.

(24)

C. Efects of Noise on Edge Location The effects of noise on the edge location determined by the operator will now be considered. The bias correction is not considered in the theoretical noise analysis. Incorporating the correction given in (15) into a noise model would complicate the analysis significantly. Assume additive, independent, identically distributed, Gaussian noise is added to the pixel gray values of a sampled ideal edge. The calculated length ( 5 ) now becomes a random variable, i.e.,

The Gaussian random variables gN and go can now be parameterized,

giy

3(M2 f

A2)

- (MO

+ A())
(25)

= 3M2 - MO f 3A2 - fio

where
c1.N

= 3M2 - M O = a2 r = l [3c2(i) - co(i)12

(26) (27)

0;

and go = 2(M, Analysis is simplified if the length is considered to be the quotient of two random variables, gN and go, such that where
PD

+ A,)

= 2M1

+ 2Al

(28) (29)

2M1

gN = 3m, - mo and go
K

= 2ml.

(19)

The moments of a discrete function f ( i ) are given by

Mp =

r=l

.xc p ( i > f ( i )
+ a(i)

(20)

where c, ( i ) is the pth order moment weighting for the pixel region indicated by i. The moments from a sampled, noisy edge, f(i) = f ( i )

The random length (18) can now be viewed as the quotient of two, nonzero mean Gaussian random variables. To completely parameterize this relationship, the covariance between the numerator and denominator random variables must be determined. The summation of (20) can be viewed as the inner product of a row and a column vector. The numerator and denominator random variables

LYVERS et al.: SUBPIXEL MEASUREMENTS USING AN EDGE OPERATOR

1297

can be modeled as
gN =

where
dTR

+ b,
l), . *
*

= CTR

+d

(31)

where
d T = [3c2(1) C T = [ 2 c l (l ) ,
RT =

erf (x) = - exp ( - y 2 / 2 ) dy. G l s O Note that if ,uN = pD = 0 the above density reduces to a Cauchy density as would be expected. Fig. 4 shows the density functions for edges located at the center of the window and at f 1 .O pixels from the center of the window where the window is 5 pixels long and the signal-to-noise ratio is 30 dB. Note that the densities for 1 = +_ 1.O pixel are not centered around 1 .O. This shift is due to the determinsitic error. The mean shift due to noise and standard deviation of the edge location estimation versus true edge location for SNRs of 30 dB, 40 dB, and 50 dB are shown in Figs. 5 and 6 . These values were derived by numerical integration from the density function in (35) using values of uN and uD from (24) and values of p N and pD obtained empirically from (26) and (29). Most notably, the mean error shows virtually no bias due to noise for lengths 1 .O pixel from the center of the window. The spatial moment-based edge operator was applied to noisy data in an empirical test. The data consisted of zeromean Gaussian random noise added to square aperture sampled edges. The window size was 5 pixels and the sampled edges location was vaned from - 1.5 to 1.5 pixels in 0.05 pixel steps. The parameter h was set to 100 while h k was set to 200. For each edge location, 10,000 different noise cases were added to the sampled edge. Although 10,000 may seem like a large number of cases, the agreement between the empirical results and the theoretical results improved as the number of cases increased. The sigma of the noise samples added to the edges is determined by the signal-to-noise ratio as [ 121:

CO(
*

, 3 C 2 ( k ) - cO(k)]

,2cl(k)]
*

[R( l ) , R(2),
- M O

,fi(k)]

b = 3Mz d
=

2M1.

The expectation of

g N g Dis

E { gNgD}= E ( (dTR

+ b)(E.f + d ) } = E{nT2i;Z + dnf + bcT.f + b d } = E { d T f X T C } + bd = d T E { R . f T } E bd + = d T C a i + bd.

E(
gNgD} -

(32)

The covariance is given by

(gNgD) =

E { g N } E( gD}
-

(33) Since the moment weightings are implemented using a symmetrical window centered at zero, is even-valued and C is odd-valued. Thus, their inner product, d T C , is zero. If the two Gaussian variables are uncorrelated, then they are necessarily independent. Therefore, the length equation is the quotient of two nonzero independent Gaussian random variables. For the special case when each random variable has zero mean, the random variable f i s Cauchy. This case arises only when the deterministic part of the window input is a flat field of zeros. In general, however, both random variables have nonzero mean. A closed form solution of the density function for the length variable does not exist. An integral form of the density [14] has the form
IYI

=~CU;

+ bd

bd =

~CO:.

SNR

k 20 log10 - dB
U

(36)

(34) It should be noted that the means of the Gaussian random variables are determined by the actual position of the edge, 1. After a fair amount of mathematical manipulation, the above integral can be expressed in terms of an error function as

where k is the edge height difference and U is the standard deviation of the additive noise. The empirical mean and standard deviation of the edge location estimation versus true edge location for SNRs of 30 dB, 40 dB, and 50 dB were also calculated. The empirical results matched the theoretical results within a maximum absolute error of less than 1 percent and an average absolute error of 0.05 percent verifying proper modeling of the system. Because of this close agreement, the empirical mean and standard deviation plots are not shown here. Figs. 7 and 8 compare the performance of the spatial moment operator and the gray level moment operator. Fig. 7 shows the empirical RMS edge location estimation error versus true edge location for SNRs of 30 dB, 40 dB, and 50 dB without table lookup correction. Fig. 8 shows the empirical RMS error when a bias correction table is used.

1298
7.0

lEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. VOL. I I . NO. 12. DECEMBER 1989

6.0]

0.0 -1.5

-1.0

-.M I

0.0

.M

1.0

1.5

Edge Translation, pixels Fig. 4. Theoretical edge translation estimation density function for 1 = -1.0 pixels (dotted), 1 = 0.0 pixels (solid), 1 = 1.0 pixels (dashed), and SNR = 30 dB.

Edge Translation, pixels Fig. 7. Empirical RMS edge translation estimation error for the gray level operator (solid) and the spatial moment operator (dashed), both without correction tables, for SNR's of 30 dB, 40 dB, and 50 dB.

-Y
-3.0

4.0J

-1.5

-1.0

-.M

0.0

.M

1.0

1.5

Edge Translation, pixels Fig. 5 . Theoretical mean translation estimation shift due to noise for SNR's of 30 dB (solid), 40 dB (dashed), and 50 dB (dotted).

Edge Translation, pixels Fig. 8. Empirical RMS edge translation estimation error for the gray level operator (solid) and the spatial moment operator (dashed), both with correction tables, for SNR's of 30 dB, 40 dB, and 50 dB. TABLE I EDGE LOCATION ESTIMATION ERROR, BOTHDETERMINISTIC IN THE AND PRESENCE OF ADDITIVE GAUSSIAN NOISE FOR -0.5 I 1 I0.5 PIXELS

.I4 -

.13-

I
Operator Gray Level Moment Gray Level Moment w/Table Spatial Moment

Edge Location Error (hundredthsof a pixel) Deterministic In Noise for an SNR o f

Max
11.62

RMS

50dB
8.19 1.61

4OdB

30dB

8.18

8.25

8.90

3 : 3
m

3.66 0.00

1.56 0.00

1.94
1.91

4.07
6.04

0.60

.rn..-..-

.m
.01-

- _ - _---..~..._....._.__-111. TWO-DIMENSIONAL OPERATOR EDGE The one-dimensional operator presented in the last section can be extended to two dimensions by multiple applications to the rows and/or columns of two-dimensional edge data, and then fitting a line to the calculated edge locations. One possible problem with this idea is that, even for step edges, there is an effective blur which is a strong function of orientation. Also, an a priori estimate of the orientation is needed to determine whether to apply the operator to the rows or the columns. This implementation yielded less than impressive results. Even when

00 1 .

Table I summarizes the performance of both operators with and without bias correction tables. Maximum and rms error for the noise-free (deterministic) case and also rms error for SNR's of 50 dB, 40 dB, and 30 dB are given. The range of I used in each measurement was -0.5 II I0.5 pixels.

LYVERS et al.: SUBPIXEL MEASUREMENTS USING A N EDGE OPERATOR

1299

using the one-dimensional bias correction table, the edge translation estimation error was as high as 0.12 pixels for ideal data, and the edge orientation error is as high as 6.2". A plot of edge translation error versus true translation and orientation is shown in Fig. 9. For these reasons, this implementation was abandoned. Next, the derivation of the spatial moment edge operator in two dimensions is considered. Many of the concepts and solutions are simple extensions of those found in the one-dimensional case. A length equation is determined from the two-dimensional moments of a continuous two-dimensional edge. As in the one-dimensional case, a bias correction table is used to remove the effects of pixel quantization. Theoretical noise analysis on the angle of orientation is compared to empirical results. Lastly, a two-dimensional version of the Tabatabai, the Hueckel and a modified version of the Huertas and Medioni edge operators are compared to the spatial moment operator.
A . Derivation of the Two-Dimensional Edge Operator A continuous two-dimensional edge specified by h, k ,

Fig. 9 . Edge translation estimation error for multiple application of 1-D edge operator to 2-D edge data.

I , and 8 is shown in Fig. 10. The first three parameters are as before. The additional parameter 8 specifies the angle the edge makes with respect to the y-axis. The edge is defined to lie within the unit circle. The moments of a two-dimensional function f (x, y ) are given by
Mpq =

Fig. 10. Two-dimensional ideal edge model characterized by h , k , I , and 0.

It is desired, therefore, to rotate the moments by the angle 8 such that Mhl = 0 (45 1 i.e., the edge will lie parallel to the y-axis. The value of 8 can be found from (42) and (45), Mhl = MO,cos 0 - M l o sin 8 = 0 (46)

XPY4f(X,

Y) d h. Y

(37)

A closed moment set of order n is closed with respect to the operations rotation, translation, and scale change and consists of moments of order n and lower. A rotation of the circular window by -8 will align all moments containing edge information along the x-axis. This is done to reduce the dimensionality of the edge problem. The parameters h, k , and 1 can be solved along the x-axis independent of 8. A rotation of the window by an arbitrary angle 4 transforms the original moments as specified by

MO I (47) MI0 To obtain the rotated moments, the following relationships can be used:

= tan-' -.

cos

(e) =

MI 0 JMil + M:o

sin (8) =

M I O JMil + M?O.
(48)

(sin 4 ) M&J = M ,

qfr-s
Mp+q-r-s,r+s.

(38) (39) (40) Mo2 (41 1 (42)

For the moments up to order two we have: Mio = cos 4MIo + sin +Mol Mio
=

These values can be used to obtain the rotated moments by substituting them into (38): M&
= =

M ,

(49) (50)

M;,

JM& M : ~ +

cos2 4Mz0

+ 2 cos 4 sin 4M1, + sin' + cos 4Mol

Mhl

= -sin 4Mlo

Mh2 = sin2 M2, - 2 cos 4 sin +MI1+ cos2 4MO2 (43) Mil = sin 4 cos 4(MO2- M20)

+ cos^ 4

sin2 4).

(44)

To obtain the rotated moments all moments of second order and lower are needed. Therefore, six moments Moo, Mlo, Mol, M I I ,MZo,and MO2must be calculated from the data. These moments are estimated by correlating the elements in the window with a mask. Each mask represents

1300

I E E E TRANSACTIONS O N PATTERN ANALYSIS A N D MACHINE INTELLIGENCE. VOL. 11. NO. 12. DECEMBER 1989

the value resulting from performing the integration of (37) over each pixel assuming f ( x , y ) is constant over that pixel. The circular limits are included in the integration. The region of this integration for a window size of 5 X 5 pixels is shown in Fig. 11. Note that some of the pixels lie along the circular boundary while others lie entirely within it. The resulting set of masks is shown in Fig. 12. Solutions for h , k , and 1 can be found in a manner similar to the one-dimeilsional analysis. Expressing the rotated moments in terms of h , k , and 1 : M&
=

1-, 1
-

hdydx

+2

1,
l m

kdydx

hn 2

+ k n - k sin-' 2

1-k

Fig. 11. A circular region defined on a 5 x 5 pixel matrix.

,0219 .I231 ,1573 .I231 ,0219 ,1231 .I573 ,1231 .0219 .I600 ,1600 ,1600 ,1231 ,1600 .I600 .I600 ,1573 .I600 ,1600 ,1600 ,1231 ,1231 ,1573,1231 ,0219

Mi0

JT--;i

hxdydu m

+2

1,' S

d i 7

-.0098 -.a352 .oooO ,0352 .0098 -.0352-.0256 .M100 ,0256 ,0352

kxdydw

.oooO .oooO .oooO .oooO .oooO

,0352 ,0256 .oooO -.0256 -.0352 ,0098 ,0352 .ooM)b.0352 -.W8

= 3 :

-.a147-.0469.oooO ,0469 ,0147 .m,0640 .0933 -.0933-.m -.1253 -.&WO .WO ,0640 ,1253 -0933 -.0640 . m.0640 ,0933 ,0147-.0469 .oooO ,0469 .0147 Mlo Mask .0099 ,0719 ,1019 ,0719 ,0194 .0021 .a194 .0099 ,0277,0021 ,0277 ,0719 ,0277BO21 .0277 ,1019 .0277 ,0021 .0277 .0719 ,0099 ,0194 ,0021 .a194 ,0099 Mm Mask

-.0469

-.m -.0640 -.W-.0469 -.0147 -0933 -.I253 - E 3 3 -.0147

.0099 .0719 ,1019 ,0719 .0099

.oooO .oooO .oooO .oooO .oooO

,0147 ,0933 .I253 .0933 ,0147 ,0469 .0640 .0640 .0640 .0469

,0194 ,3277 .0277 ,0277 ,0194

,0021 .0021 ,0021 ,0021 .IO21 ,0194 .0277 ,0277 ,0277 ,0194

.0099 ,0719 .I019 ,0719 ,0099

h k kJ--= - n + - n + - ~ (1 - 1 2 ) 3 4 8 2

Fig. 12. The set of six masks used for moment operator with ws = 5

The equations (53), (54), and (55) can be combined to solve for 1:

Once 1 is determined, k and h may be obtained from (54) and ( 5 3 , respectively,

k =
2

3w0

(57)

Fig. 13. Edge translation estimation error for the spatial moment operator with a window size of 5 pixels.

1 h = - [2M& 27

k(n

2 sin-' 1 - 21-)].

B. Bias Effects Due to Pixel Sampling As in the one-dimensional case, the edge operator is derived using a continuous two-gray level model. Again, the model does not allow for sampling effects present in digitized data. In general, the moments of the sampled edge do not match those of a continuous edge and thus an error arises in the edge location. Fig. 13 shows the bias

error of the two-dimensional edge operator as the edge is varied from 0 to 1.5 pixels and as the edge's orientation is varied from 0" to 45" in a circular window of size 5 pixels. The error has even symmetry about the plane 8 = 45 " . This symmetry also exists about the planes 8 = 0" , 90", 135", etc. Although not shown here, it was empirically verified that the error plot has odd symmetry about the plane 1 = 0. Due to this symmetry, performance over all orientations can be measured with the range of 0 5 8 545". The bias error can be derived theoretically. However, moment integration over an edge with an angle not aligned

LYVERS et al.: SUBPIXEL MEASUREMENTS USING A N EDGE OPERATOR

1301

with the rectangular sampling grid is difficult. Therefore rather than solving the bias error for an arbitrary angle, the bias error is derived for an edge oriented parallel to the y-axis only. In this situation only three gray levels are present ( h , A k , k ) in the pixel samples. The moments are

where

p=-

Ak k'

0 5 p 5 1 .

(63)

11,
+2 +A
-

JG3
J0

il: 1 7
- l2k [ 1 2 m
-

dY dr +

!:y 1
0

AT7

Ak dY dx

The length to the edge given a three-1eve1 (edge oriented parallel to the y-axis) with a square aperture function is identical to that obtained for the one-dimensional case (15). Therefore, the bias error is

k dy dr
-

= ah

+ k [2
1,-

sin-'
-

P(P

1 ) ( & - 11)

[./(1-I:,J+0

m]
(64)
= 0.

+ sin-'

l2

(1 -

0 ) -

sin-' I ] ]

(59)
The bias error is zero when

Akxdydx

+2

1 7
+2

p(1

0)

[m1-

(65)

kx dy dx

(60)

hx2dydx

I,,

Io

Akx2 dy ah

+2

kx2 dy d x

- -

1 sin-' 12] 2

1 ; sin-'
L

(61 ) Substitution of Moo, MIo,and M20 into the length equation results in (1 l!M

P ) 1 2 4 i T q + 61-'
P ) r n + P r n

The above result is true when P = 0 or /3 = 1 which occurs when the edge is situated on the pixel boundaries and the three-level edge is reduced to a two-level edge. The other circumstance occurs when l2 = - lI. As in the one-dimensional case, this occurs when the edge is near the center of the window. For a window of diameter 5 pixels, 1' and l2 will equal -0.2 and 0.2, respectively. A plot of estimated edge translation versus I and 0 (not shown) shows that it is monotonically increasing with 1 and hence it should be possible to generate a bias correction table. Such a table was generated and is shown in Fig. 14. Note that it is a correction term versus estimated translation and estimated orientation. Using this bias table and bilinear interpolation, the maximum corrected estimated edge translation error for 1 I \ 5 0.5 pixels is less than 0.0045 pixels. The two-dimensional gray level moment operator and the Hueckel operator have been implemented and their bias values are plotted in Figs. 15 and 16. The gray level moment operator bias has a large oscillating bias near 0" and near 45O . Other angles show oscillation, but at a much reduced magnitude. Since the bias is smoothly varying and of moderate value, a bias table can be used to reduce the bias. Using this bias table and bilinear interpolation, the maximum corrected estimated edge translation error for 11 1 I pixels is less than 0.052 pixels. The Hueckel 0.5 operator's bias, on the other hand, shows some very serious problems at isolated location-angle positions. The amount of bias at these positions range as high as 1.6 pixels error. Due to the high bias, the actual length versus calculated length relationship is not monotonic. Therefore, a bias table cannot be implemented for the Hueckel operator. The Huertas and Medioni operator detects edges as zero crossings in the convolution of the image with LOGmasks. Subpixel location of the zero crossings is obtained by using a 2-D second order polynomial facet model on

1302

IEEE TRANSACTIONS ON PATTERN ANALYSIS A N D MACHINE INTELLIGENCE. VOL. 1 1 . NO

12. DECEMBER 19x9

2
m

Fig. 16. Edge translation estimation error for the Hueckel operator with a window size of 5 pixels. Fig. 14. Edge translation bias correction table for the spatial moment operator with a window size of 5 pixels.

d
Is
,0111

E : 2

.,Mm

-.lo48

Fig. 17. Edge translation estimation error for the Laplacian of a Gaussian operator using the facet model for subpixel resolution. The window size is 13 pixels.

Fig. 15. Edge translation estimation error for the gray level moment operator with a window size of 5 pixels.

tionf(i3 j

the moments are

K L

as

Mp4 =

1 = 1

C JC =I

cpq(i,;)f(i,j).

(66)

3 x 3 regions of the filtered image. The resultant polynomial is used for image resampling of 2 X or 3 X resolution improvement. zero crossings to pixel accuracy are found in the resampled image, thus providing subpixel accuracy in the original image. Our implementation is similar except that the zero crossings are found directly from the second order polynomial, and a straight line is fit to them. The edge location and orientation are calculated from the line parameters. The result of this is shown in Fig. 17. Note that the operator requires a window size of 13 X 13 so that its performance near comers would not be good. C. Effects of Noise on Edge Location For the two-dimensional noise analysis, additive, independent identically distributed Gaussian noise was added to a sampled ideal edge. For a discrete edge func-

The noisy edge may be

as

J;(i,j) = f and the moments as

K L Cpq(

fipy =
=

C C j = l j=l
K L

i, j

i= I j = I

c c c&,;

The rotated moments M i q ( 6 ) are M&( e )

=

Mm

=
'

C Cm(i,j)f(i,j)
J

(69)

LYVERS et al.: SUBPIXEL MEASUREMENTS USING AN EDGE OPERATOR

1303 TABLE I1 EDGELOCATION ESTIMATION ERROR, BOTHDETERMINISTICIN THE AND PRESENCE OF ADDITIVE GAUSSIAN NOISE

Although the moments (68) are Gaussian random variables, the rotated moments, with the exception of M&,, are not. This can be seen from (69), (70), and (71). The expression for Mio is quite complex. Due to this complexity, theoretical noise analysis for the length estimate was not done. Instead, empirical noise analysis similar to that of the 1-D case was done. The number of cases was limited by computer time and was set to 1000. The sigma of the noise samples was again determined by (36) and SNR's of 50 dB, 40 dB, and 30 dB were used. Empirical mean-shift, standard deviation, and rms data versus edge location and orientation were generated. These tests were performed on the spatial moment operator with and without a bias correction table. For comparison, these tests were also performed on the 2-D gray level moment operator [lo] both with and without a table, the Hueckel [7]-[9] operator, and the LOG operator. Each of these data generate a 3-D plot. Due to space constraints, however, none of these plots will be shown here. Table I1 summarizes the performance of each operator. Maximum and rms error for the noise-free (deterministic) case and also rms error for SNR's of 50 dB, 40 dB, and 30 dB are given. The range of I used in each measurement was 0.0 5 1 I0.5 cos 8 pixels. This range of 1 corresponds to that of the given window being the closest to the edge. Note that the use of a bias correction table markedly improves the performance of both the gray level and the spatial moment operators.

* Some sample windows did not convergeand ae not included in these slalisucs. r

where p y is the mean of the numerator, p, is the mean of the denominator, and U is the standard deviation of both the numerator and the denominator (they are equal). Using the transfomation pX cos 8 pJ sin 8 = p cos (8 - 4 ) where p2 = p: p: and 4 = tan-' ( p , / p , ) we obtain:

D. Effects of Noise on Edge Orientation

The two-dimensional moment operator also estimates an edge's orientation. This estimate is necessary to fully parameterize the edge and was investigated for its performance both in the noiseless case and in the presence of noise. Theoretical noise analysis of the angle estimate is straightforward. The angle is determined by the first order moments in x and y and is defined by (47). As before, the moments fioland A?,, are random with Gaussian distribution when Gaussian random noise is added to the data. It can be shown [13] that the moment weightings cause Mol and fila to be independent of each other. The angle is simply a quotient of two independent Gaussian random variables with the added transformation of arc tangent. The density function for the angle [13] is given by:

From (73), it can be seen that the density function is symmetrical about C#J = tan-' ( p y / p X )so that the mean value of 8 is tan-' ( p , / p X ) . Empirical tests on the orientation estimation using the parameters from above were also performed. Again, the spatial moment and the Hueckel operator were also tested. Table I11 summarizes these results.

exp

( p X cos

+ p v sin 8 )
2a2

where
4

erf (x) =

G 1O
PX

exp ( -y2/2) dy

E. Nonideal Edges and Nonsquare Pixels In practice (due to optics, etc.) edges may not be accurately modeled as a step, but appear to make a sloped transition from one gray level to another. Bias correction tables can be designed to handle such edge profiles. For a linear ramp edge there is the additional parameter of ramp width. Bias correction tables for ramp widths of 0.35 pixels and 0.70 pixels were generated and are shown in Figs. 18 and 19, respectively. Note the similarity to Fig. 14 and how the shape changes as the width of the ramp edge increases. Up to this point square pixels have been assumed. The problem with this assumption is that imaging devices (CCD cameras, etc.) often have pixels that are not square but are rectangular. Assuming the pixels to be square when they are actually rectangular can cause edge translation estimation errors as well as edge orientation estimation errors. For example, if an edge were to pass through the upper left comer and the lower right corner of the middle pixel, an operator assuming square pixels would estimate an edge orientation of 45 . If the pixels
O

1304

IEEE TRANSACTIONS ON PATTERN ANALYSIS A N D MACHINE INTELLIGENCE. VOL

II. NO. 12. DECEMBER 1989

TABLE 111 EDGEORIENTATION ESTIMATION ERROR, BOTHDETERMINISTIC I N THE AND PRESENCE OF ADDITIVE GAUSSIAN NOISE
Edge Angle EHOI(degrees)

Operator
Gray Level Moment Gray Level Moment wnable Hueckel* Laplacian of Gaussian Spatial
1

Deterministic Max

I
I

RMS In Noise for an S N R of

5OdB

RMS
0.355 0.0078 0.335

4odB
0.458 0.292 0.335 2.55 0.458 0.292

30.981 0.924 0.335 2.80 0.981 0.923

0.996
0.125 0.753 3.70

0.367 0.093 0.335 2.52 0.367

I

2.52
0.355
1

0.996
I

Spatial wnable
-

0.0058

0.0023

0.092

* Some sample windows ~d

m

no1 converge and are n itwluded m rhese staosucs a

X .-

Fig. 20. A circular region defined on a 5 ratio is 4 : 3.

5 pixel matrix. The aspect

Fig. 18. Edge translation bias correction table for the spatial moment operator with a window size of 5 pixels, and an edge ramp width of 0.35 pixels.

fusing for quantities measured in pixels such as edge translation. Such parameters will be measured in x-pixels to avoid confusion. The masks resulting from integration over the circular region of Fig. 20 are shown in Fig. 21. Similar to before, the moment operator using these masks was tested on a synthesized ideal step edge. The routine used to generate the edge was modified to allow for a variable aspect ratio. Using the same parameters as before, an error plot was generated and is shown in Fig. 22. A view of the plot rotated 180" is shown in Fig. 23. Note that the plot now does not possess symmetry about the plane 8 = 45 ". The error for 8 = 0" is zero for 0 I1 5 0.5 x-pixels and for 1 = 0.5 x-pixels, but for 8 = 90" the error is zero for 0 I1 5 0.375 x-pixels and for 1 = 1.125 x-pixels. However, 0.375 x-pixels is equal to 0.5 y-pixels and 1.125 x-pixels is equal to 1.5 y-pixels. This occurs because the distances to pixel boundaries are different for edges of orientations of 0" and 90" as can be seen from Fig. 20. Although not shown here, bias correction tables were generated for the 4 : 3 aspect ratio for step edges and ramp edges with ramp widths of 0.35 and 0.70 x-pixels.

Fig. 19. Edge translation bias correction table for the spatial moment operator with a window size of 5 pixels, and an edge ramp width of 0.70 pixels.

were rectangular and the ratio of x-pixel size to y-pixel size were 4 : 3, then the true orientation would be 53.1 " resulting in an error of 8.1 O . The moment-based operator was modified to accommodate rectangular pixels. Since the circular region was defined on a square pixel matrix (Fig. 1l), this matrix can be modified to allow for rectangular pixels. The ratio of x-pixel size to y-pixel size are referred to as aspect ratio. A circular region defined on a 5 X 5 pixel grid with an aspect ratio of 4 : 3 is shown in Fig. 20. Note that the window diameter is 3.75 x-pixels or 5.00 y-pixels. This difference in pixel size can be con-

F, Effects of Gray Level Quantization Up to this point, the edge data gray values have not been quantized, but have been allowed to take on an infinite number of values. All computer simulation programs used double precision floating-point numbers to simulate this. Since subpixel operators use gray-level resolution to obtain subpixel edge location, the number of gray values has a direct effect on edge location accuracy. Actual image data is quantized to a finite number of levels, typically 256, ranging from 0 to 255. Consider an edge sample oriented at 8 = 0". For this angle, each row in the edge sample is identical. If the edge values are quantized (rounded) to integer values, they will remain constant for variations in 1 of up to one over the number of gray levels in the edge height. This places a theoretical limit on edge location accuracy of the best possible edge operator as one part in twice the number of gray levels between edge heights. The edge estimation error will be

LYVERS er a l . : SUBPIXEL MEASUREMENTS USING AN EDGE OPERATOR

1305

.Woo ,1202 ,2069 .I202 .Woo

,0431 ,2133 .2133 ,2133 .0773 .2133 ,2133 .2133 ,0431 ,2133 .2133 .2133 .oooO ,1202 ,2069 .1202 Mm Mask

.OK31 ,0773 ,0431 .Woo

-.0128 4 4 5 5 .ME0 0 4 5 5 ,0128

.oooO -.0405 .oooO ,0405 .oooO

.oooO .oooO .oooO .oooO ,3300 ,0128 .0455 .Woo -0455 -.0128 .oooO ,0405 .oooO -.0405 .oooO
MII Mask

-.0373 -.1138 .oooO ,1138 .0373 -0693 -.I138 .oooO ,1138 ,0693 -.0373 -.1138 .oooO ,1138 ,0373

.oooO -.OS58 .oooO . O S 8 .oooO

.oooO .oooO .oaoO .oooO .oooO

-.0149 -.OS53 -.0853 -.0853 s.0149 .Woo s.0885 -.1643 -.OM5 .oooO
Mol

.oooO ,0885 ,1643 ,0885 .oooO ,0149 .OW3 ,0853 ,0853 ,0149

.oooO -.OS58 .oooO .0558 .oooO Mi0 Mask

Mask

.oooO ,0279

,0048 ,0279 ,0324 ,0657 ,005 1 ,0657 ,0624 ,0657 ,0051 ,0657 .0324 ,0657 ,005 1 ,0657 .oooO .0279 ,0048 ,0279 Mm Mask

.oooO

,0324 3624 ,0324

.oooO ,0661 .I331 ,0661 .oooO

,0055 .0370 ,0370 ,0370 ,0055 ,0010 ,0028 ,0028 ,0028 .0010 ,0055 ,0370 .0370 ,0370 ,0055 .oooO .0661 .1331 .0661 .oooO % Mask

Fig. 21. The set of six masks used for moment operator with aspect ratio of4:3. Fig. 23. Mirror image of Fig. 22.

G. Selecting a Confidence Measure

As with many other edge operators, some type of confidence testing is necessary to determine that the estimated edge parameters are accurate. Since the edge operator was developed assuming a single straight edge in its window, it might not give correct results for other features such as lines and comers. Empirical tests show that the results for lines and comers indeed are not accurate. The image shown in Fig. 24(a) was generated to test a confidence measure. It includes comers, ideal step edges, and lines, each with a step height of 100. A good confidence measure would be able to reject the comers and lines while detecting the step edges reliably. Magnitude thresholding is a well known method of edge detection, but often yields poor results. Fig. 24(b) shows the results of magnitude thresholding at a gray level of 20. Note that the results are very poor. Fig. 24(c) shows the results of a thinning in the direction normal to the estimated edge orientation by retaining only the pixel closest to and also within one-half of a pixel from an edge. The results of this are better, but are still unacceptable. Another method of confidence testing is to compute the variance of an ideal step edge having the same 1, 8, and k (step height) as those estimated and compare it to the actual variance of the image data. Agreement between these two values would indicate a good edge. Comparison of the calculated and actual variance proved to be a very poor confidence measure. In most cases (even on comers and lines) the two were very close, having no apparent dependence on what type of feature was in the window. Obviously, another confidence measure is needed. Notice that Mh2 is not used in the calculation of edge parameters. This value (47) can be compared to that of an ideal step edge with the same 1, 8, and k (step height) as those calculated. This confidence measure was good at identifying corners but failed to reject lines. Another method is to compare the calculated edge parameters to those of the next closest window. A consistency in the two sets of edge parameters would indicate that the results were accurate. The edge parameters of orientation, location, and step height were used. A deviation in the two orientations of 3", the step heights of 15 per-

Fig. 22. Edge translation estimation error for the spatial moment operator with a window size of 5 pixels, with an aspect ratio of 4:3.

a sawtooth function of true edge translation, so the RMS edge error will be one part in 2 h times the number of gray levels between the two edge heights. This limitation was analyzed assuming 8 = 0", so it may not be valid at other orientations. In fact, it can be seen that at other angles the rows will not be identical, so the limit of accuracy may be better. Empirical tests of gray value quantization on edge location accuracy were performed for the spatial moment operator as well as for the gray level moment operator. These tests were performed using integer valued edge data with edge step heights of 100, 50, 20, and 10. Also, the effect of ramp edges were studied using ramp widths of 0.0 (step edge), 0.25, 0.50, 0.75, 1.00, 1.25, and 1.50 pixels. The maximum and rms edge location errors for 0 II I1.5 pixels (as a function of 8 ) were measured. For the spatial-moment operator and a step edge, the empirical results for 8 = 0" were exactly as predicted for the best possible edge operator. The error showed a strong dependence on 8, however. The rms error was lowest for 8 = 1" being less than one-third that at 8 = 0". The rms error over all angles was approximately one-half that at 8 = 0 " . The effect of a ramp edge was a small increase in error which was almost linear with ramp width, being one and one-third that of a step edge for a ramp width of 1.50 pixels. The gray level moment operator did not perform quite as well having an rms error that was two and onehalf times that of the spatial with 100 gray levels.

1306

Ik.EE TRAKSACTIONS ON PATTERN ANALYSIS .AKD hlACHIKE I V T E L L I G t N C F . VOL

I I . NO

17. DECEMBER 1989

cation 1, edge heights h , k , angle of the edge 8, and a confidence result.

I. Subpixel Edge Location in Real Image Data The edge operator developed in the last section was used to locate edges in real image data to subpixel accuracy. The operator was shown to have an accuracy of better than 0.05 pixels for the image data used. A digitized image of a backlighted rectangularly shaped piece of 0.0625 steel was obtained using an Eikonix scanner. The rectangle was precision machined to 2 by 4 inches within two ten-thousandths of an inch in each dimension. The image digitizer uses a 1-D photodiode array which is mechanically positioned to scan in the y direction. The image obtained from this scanner had a resolution of 1024 x 1024 8-bit pixels. A test image was genFig. 24. (a) Image for confidence test. (b) Result of magnitude thresholding. (c) Result of distance thinning. (d) Result o f confidence measure erated by averaging over 2 x 2 regions of the original based on consistency in estimated edge parameters. image resulting in an image with a resolution of 512 x 512 8-bit pixels and is shown in Fig. 25. The rectangle cent, and edge location of 0.15 pixel or less was consid- orientation is approximately 2 O from the coordinate axes ered acceptable. The result of this confidence test is shown of the digitizer. A blow-up of the upper left comer of Fig. in Fig. 24(d). Notice that only the step edges are detected 25 is shown in Fig. 26. Note how the pixels appear to fit and the comers and lines are rejected. The confidence the square aperture sampled model. The signal-to-noise measure was also tested on the image of 24(a) with white ratio in this image exceeded 40 dB, therefore noise was Gaussian noise samples added. The signal-to-noise ratio not the cause of inaccuracies in this data. One test of the was set to 30 dB. The result was the same as that without operators subpixel edge location accuracy is to apply it noise, showing its robustness in moderate signal-to-noise to a straight edge and check the subpixel location as the operator moves along the edge. The operator was applied ratios. to the left vertical edge of Fig. 25. At each pixel along H , Implementation the edge the distance to the edge from each window center In summary, to implement the two-dimensional edge along the x direction was estimated. Since the operator operator the following steps must be performed to obtain calculates a normal distance to the edge, the estimated edge orientation was necessary to calculate this. The acprecision edge location. I ) Generate circular moment masks. Section 111-A de- tual edge location was calculated by adding this subpixel scribes the process for generating circular moment masks distance to the pixels x location. This edge location for discrete data. should show a linear change along the edge. This test was performed using the ideal step edge bias 2) Generate bias table. Generation of the bias error tacorrection table shown in Fig. 14. The results were unble is discussed in Section 111-B. Steps 1 and 2 need to be performed only once and can satisfactory. Errors of up to 0.08 pixels occurred. The be done off-line. However, the moment masks and bias operator underestimated the subpixel edge distance. A table may need to be changed for different window sizes, closer look at the image data of Fig. 25 shows that the edge is more closely modeled as a ramp edge. The test shapes, or edge blur. 3) Compute the two-dimensional moments. The mo- was performed again using a bias correction table for a ments of the window are computed ( M o o ,M o l , Mlo, Mil, ramp edge of width 0.7 pixels. Sample results from this MO2, The moment masks permit this to be accom- test are shown in Table IV. Thc most significant information is found on the line of Table IV where the x loplished by correlation with the edge data. 4) Compute 8. This is accomplished with the two-di- cation of the window center shifts one pixel to the right and the subpixel fractional estimate changes sign from mensional moments and (47). 5) Rotate the moments. Rotate the two-dimensional maximum positive to maximum negative. This position offset varied by at most -0.02, +0.01 pixels over the moments by 8 , (49), (50), and (51). 6) Compute I , k , and h. These parameters are obtained entire image edge. This test was also performed on the three remaining edges and gave similar results. This edge using (56), ( 5 7 ) , and (58). 7) Correct the bias. The length estimate ( 1 ) is cor- error of course depends on the window size used. For a rected with bias table. The angle estimate may also be window size of 9 X 9, the error was kO.01 pixels; for a 3 x 3 , it was k 0 . 0 2 pixels. corrected using a similar table. Although an accurate operator would have a constant 8) Measure conjdence as per Section III-G. Steps 3-8 are repeated for each window of data. The line-to-line difference, this in itself does not guarantee result at each pixel location will be a subpixel edge lo- overall accuracy because it fails to show cumulative error.

LYVERS er al.: SUBPIXEL MEASUREMENTS USING AN EDGE OPERATOR

1307

TABLE 1V STRAIGHT EDGELOCATION RESULTS REALDATA(WINDOW ON SIZE= 5)

181
I

187

I -178.176 I
I

-0.4497
I

180.550

I
I

0.033 0.035 0.036

181 181

189 190

-177.849 -177.895

-0.3784 -0.3422

180.622 180.658

Fig. 25. Digitized image of rectangular steel piece.

-""L_
-.12s -.1M

im

147

174

mi

228

254 281 308 33s 362 389

Line number

Fig. 27. Edge location estimation error for data of Fig. 25

TABLE V STRAIGHT EDGELOCATION RESULTS REALDATA(ASPECT ON RATIO : 3) 4 Fig. 26. Blowup of upper left comer of Fig. 25.
x pixel

90

I y pixel 1 theta est. I fract. x est. I total x est. I xdiff prev. line I 89 1 -178.151 I 0.3787 1 90.379 I 0.025
90 -178.380

A est which shows both errors is to fit a straight line I the estimated edge locations, and then calculate the difference, on a line-by-line basis, between the estimated edge location and the position of the line. The result of this test for the image data of Fig. 25 is shown in Fig. 27. Note that the results are not as accurate as was previously thought. Fig. 27 shows accuracies of approximately k0.05 pixels. To test the operator's performance on data with rectangular pixels, a test image of aspect ratio 4 : 3 was generated by averaging 4 X 3 regions of the original 1024 X 1024 pixel rectangle image. The straight edge test of before (Table 11) was performed. Best results were obtained using a bias correction table for a ramp width of 0.37 x-pixels and are shown in Table V . Again, this test was also performed on the three remaining sides of the rectangle and similar results were obtained.

90

0.4015

90.402

0.023

J . Relationship to the Sobel Operator Although this paper has addressed subpixel measurements, the moment based operator presented can be modified for use as a general purpose edge operator. Applications of this include low signal-to-noise images where

subpixel measurements are not practical and also where subpixel measurements are not needed. For this case, only M10 and MO, need to be estimated as the orientation is calculated by (47) and the edge step height is calculated by (57) assuming 1 = 0. Setting 1 = 0 results in a sacrifice in accuracy of less than 5 percent [13]. Although throughout this paper a window size of 5 pixels is used, the operator may be implemented for any window size. Fig. 28 shows the Mlo and MO, masks for a window size of 3 pixels. The factor of 3 / 2 in (57) has been incorporated into these masks. A comparison with the Sobel reveals that the two are almost identical. The edge orientation and steu height are estimated in the same

1308
-0.1374 O.oo00 0.1374 -0.2840 O.oo00 0.2840 -0.1374 O.oo00 0.1374

I E E E TRANSACTIONS O N PATTERN ANALYSIS A N D M A C H I N E INTELLIGENCE. VOI.. 1 1 . NO. 12. DECEMBER 1989

0.1374 0.2840 0.1374 O.oo00 0.2840 O.oo00 -0.1374 -0.0853 0.1374
Mol

Mia Mask

Mask

Fig. 28. M , , and MO, masks used for general purpose operator with a window size of 3 pixels.

manner with M l o analogous to Ax and M O ,analogous to Ay. In fact, the ratio of pixel weightings is 2.067 whereas for the Sobel it is 2.0. Therefore, the general purpose moment based operator provides a method of extending the Sobel to arbitrarily large window sizes. Larger window sizes allow significantly better edge orientation and step height estimations both in the noise-free case and in the presence of noise [13]. In fact, the moment-based operator with a window size of 5 pixels in the noise-free case has an orientation estimation error of less than 1.O while its edge step height estimation error is less than 5.0 percent [ 131. Contrast this with the error of the extended Sobel (window size of 5 ) which has an orientation estimation error in excess of 2.0 [16]. IV. SUMMARY A gray-level moment-based edge operator was developed which operates on a window from a digitized image to determine the location, angle, and gray level step height of an edge. The operator can accommodate edges sampled with rectangular pixels as well as ramp edges. In the presence of noise, the operators location estimate was shown to have a relatively small bias and a small standard deviation when an edge is near the center of the window. In comparison, the Hueckel and Tabatabai edge operators were shown to have high edge location estimation bias errors. The Hueckel operator was also shown to have a large edge location estimation length variation in the presence of noise. Application of the operator to the digitized image of a backlit metal plate resulted in the location of the edge to within 0.05 pixel and the determination of the angle to within 0.3 degrees.

Erratum for A local visual operator which recognizes edges [9] -, and lines, J. A C M , vol. 21, p. 350, Apr. 1974. [lo] A. J . Tabatabai and 0. R. Mitchell, Edge location to subpixel values in digital imagery, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-6, no. 2 , pp. 188-201, Mar. 1984. 1111 A. Huertas and G. Medioni, Detection of intensity changes with subpixel accuracy using Laplacian-Gaussian masks, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, no. 5 , pp. 651-664, Sept. 1986. [ 121 A. Englander, Expanding machine vision gauging with sub-pixel techniques, Sensors-J. Machine Perception, vol. 4, no. 6 , pp. 918, June 1987. [13] E. P. Lyvers and 0. R. Mitchell, Precision edge contrast and o n entation estimation, ZEEE Trans. Partern Anal. Machine Inrell., vol. PAMI-10, no. 6, pp. 927-937, Nov. 1988. [ 141 A. Papoulis, Probability, Random Variables, and Stochastic Processes. New York: McGraw-Hill, 1965, p. 65. [I51 W. K. Pratt, Digital Image Processing. New York: Wiley, 1978, p. 498. [16] A. Iannino and S . Shapiro, An iterative generalization of the Sobel edge detection operator, in Proc. ZEEE Con& Pattern Recognition and Image Processing, Aug. 6-8, 1979, pp. 130-137.

Edward P. Lyvers was born in Fort Wayne, IN. He received the B.S.E.E., M.S.E.E., and Ph.D. degrees from Purdue University, West Lafayette, IN, in 1978, 1980, and 1988, respectively. While working toward the Ph.D. degree, he was sponsored by an RCA Zworykin Fellowship and the Engineering Research Center at Purdue. He is now employed at MIT Lincoln Laboratory, Lexington, MA. His current research interests include many aspects of computer vision and image urocessing. some of which are image acquisition, edge detection, and automated inspection using computer vision. Dr. Lyvers is an active member of Eta Kappa Nu.
I .

REFERENCES
[ l ] W. Frei and C.-C. Chen, Fast boundary detection: A generalization and a new algorithm, IEEE Trans. Comput., pp. 988-998, Oct. 1977. [2] L. G. Roberts, Machine perception of three-dimensional solids, in Optical and Electro-Optical Information Processing, J. T. Tippett et a l . , Eds. Cambridge, MA: MIT Press, 1965. [3] R. 0. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973. [4] P. J. Macvicar-Whelan and T. 0. Binford, Line finding with subpixel precision, Proc. SPIE, vol. 281, 1981. [5] R. Machuca and A. L. Gilbert, Finding edges in noisy scenes, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-3, pp. 103111, Jan. 1981. [6] R. Nevatia and K . R. Babu, Linear feature extraction and description, Comput. Graphics Image Processing, vol. 13, pp. 257-269, 1980. [7] M. H. Hueckel, An operator which locates edges in digitized pictures, J . ACM, vol. 18, pp. 113-125, Jan. 1971. [8] -, A local visual operator which recognizes edges and lines, J . ACM, vol 20, pp. 634-647, Oct. 1973.

Owen Robert Mitchell (S64-M72-SM82) was born in Beaumont, TX, on July 4 , 1945. He received the B.S.E.E. degree from Lamar University, Beaumont, in 1967 and the S.M.E.E. and Ph.D. degrees from the Massachusetts Institute of Technology, Cambridge, MA, in 1968 and 1972, respectively. In 1972 he joined the faculty of Purdue University where he served as Professor of Electrical Engineering, Assistant Dean of Engineering for Industrial Relations and Industrial Research, and an Associate Director of the NSF Engineering Research Center for Intelligent Manufacturing Systems. He spent two summers at White Sands Missile Range, where he familiarized himself with the problems of real-time video tracking and pattern recognition. In 1988 he became Chairman and Professor of Electrical Engineering at the University of Texas at Arlington. He is currently active in teaching and research in the areas of digital image processing and computer vision, shape and texture analysis, and precision measurements in images.

LYVERS et al.: SUBPIXEL M E A S U R E M E N T S USING AN E D G E OPERATOR

1309 Dr. Akey is a member of Eta Kappa Nu, Sigma Pi Sigma, AFCEA, and the Corporation for Science and Technology of Indiana.

Mark L. Akey (S84-M86) received B S.E.E ,

M.S.E.E., and Ph.D. degrees from Purdue University, West Lafayette, IN, in 1980, 1982, and 1985, respectively He is a Staff Member of DACE (Decision support systems Applied Center of Excellence), a corporate level IR&D group within Magnavox Electronics System Company, Fort Wayne, IN. In this capacity, he is manager of and specific investigator in corporate-wide research efforts as they relate to the areas of decision support systems and artificial intelligence His technical contnbutions to the group typically encompass research methodologies, concept development, and prototypical implementations. In addition to the above areas, his background and interests include the areas of computer vision, statistical pattern recognition, neural networks, and chaos theory Prior to joining Magnavox, he consulted for a number of companies, pnncipally in the area of image processing These companies included Battelle, Hughes Aircraft, ATT Electronic Photography and Imaging Center, White Sands Missile Range, I-SCAN, and Product Research Organization. These efforts included FLIR data target recognition, image compression of color images, and pattern recognition in multidimensional feature spaces.

Anthony P. Reeves (M76-SM84) received the B.Sc. degree (honors first class) in electronics and the Ph.D. degree from the University of Kent, Canterbury, England, in 1970 and 1973, respectively. He is currently an Associate Professor in the School of Electrical Engineering, Cornell University, Ithaca, NY. Previously, from 1976 to 1982, he was an Assistant Professor in the School of Electrical Engineering at Purdue University. He has held visiting faculty positions at the University of Wisconsin, Madison; McGill University, Montreal, P.Q., Canada; and Pavia University, Italy. From 1987 to 1988 he was a member of the faculty of the Department of Computer Science, University of Illinois at UrbanaChampaign. His current research interests include parallel processing and computer vision. He developed the high level programming language, called Parallel Pascal, for NASAs Massively Parallel Processor and is now working on dynamically adaptable programming environments for multicomputer systems. Vision research is centered on the macro feature approach to three-dimensional object recognition. Dr. Reeves is a member of the Association for Computing Machinery, Sigma Xi, and Eta Kappa Nu.