Basic Dimensional Metrology

Chapter 2. Measurement Errors

2.1 Introduction
The main objective of the quality control department in a manufacturing company is to
inspect the manufactured parts and ensure that they are made within the tolerance limits specified in
the drawing or a blueprint. Close tolerance will enable the parts to be assembled without difficulty
to build the final product. Precision measurement plays an important role in the process of design
and manufacture of the product by ensuring that all the parts are manufactured within the design
specifications and tolerances required.
The accuracy of measurement needed for a part depends mainly on the function of the part.
The accuracy can be affected by errors present in the measuring instrument as well as in the
measurement process. The sources of all these errors must be understood and steps must be taken
to minimize or eliminate the errors so that high accuracy measurement can be achieved.
2.2 Errors in measurement
The main objective of carrying out measurement is to enable someone to make a decision
about a product, for example, whether to accept the product, reject the product completely or send
it back for rework. This type of measurement service will not be complete if the measurement is not
carried out to the desired level of accuracy. However, no measurement can be said to be exact due
to the limited accuracy of all measuring instruments. Therefore, in any measurement we not only
have to state the dimension measured but also mention the accuracy with which the measurement
has been carried out.
For instance, if a shaft of nominal diameter 30 mm is measured using a micrometer of
accuracy 0.01 mm and the micrometer shows a reading of 30.2 mm, then the measurement should
be expressed as 30.2 0.01 mm. The errors in measurement should be minimized as far as possible
and the magnitude of error still present in the measurement should be stated, e.g.
Measured diameter = 30.2 0.01 mm
Error in measurement = 0.02 mm
Actual diameter = (30.2 + 0.02) 0.01 mm = 30.4 0.01 mm
Errors in measurement can generally be divided into two categories: (a) Controllable errors
and (c) random errors. Controllable errors are errors that can be eliminated by careful measurement.
Such errors can also be removed by correcting the measurement so that the effect of the error can
be cancelled. There are several types of controllable errors, such as catastrophic errors, alignment
errors, errors due to ambient condition and errors due to elastic deformation. These errors can be
reduced by taking the necessary precautions during the measurement.
Random errors are errors that are contained within the measurement process itself and are
difficult to be removed. Examples of random errors are scale errors, reading errors, random
fluctuations in readings due to condition of part surface etc. Because random errors cannot be
eliminated, statistical methods are necessary to analyze these errors and obtain meaningful results.
2.3. Types of controllable errors
2.3.1 Catastrophic errors
Catastrophic errors are large magnitude errors that result from errors in reading an instrument.
For instance, a reading displayed as 5.78 mm may be misread as 5.28 mm. Such errors could also
result from careless mistakes in calculations or simply by pressing the wrong button on a calculator.
8

Mani Maran Ratnam (2009)

Basic Dimensional Metrology

Catastrophic error can be easily detected when the measurement is repeated and new readings are
compared with the other readings. For instance, assume that a measurement process results in the
following dimensions on a component: 20.2 mm, 20.1 mm, 19.9 mm, 20.1 mm, 23.0 mm and 20.0
mm. In this set of measurement it is obvious that the reading having the value 23.0 mm is
significantly different from the others and this could be due to errors made by the instrument
reader. This reading can, therefore, be removed from the set of measurements before further
analysis is carried out, such as determining the average dimension.
2.3.2 Alignment error
Alignment error occurs because the instrument is wrongly aligned relative to the workpiece or
part being measured. An example of a situation that causes alignment error in the use of a dial
indicator is shown in Figure 2.1. The dimension of interest is D, but due to misalignment the dial
indicator gives a reading L which is larger than D. The error e in the measurement due to the
misalignment is, therefore,
Error , e = L D = L L cos = L (1 cos )

(2.1)

For example, if the angle due to misalignment = 10 and the dial indicator shows a reading on
4.52 mm, then the error e = 4.52(1 cos10) = 0.069 mm.
Since this type of error is a function of the cosine of the angle of misalignment, hence it is
also known as cosine error.

Dial indicator

L D
b
a

D
b

Figure 2.1. Misalignment error.

2.3.3 Calibration error

The actual dimensions of standards used for calibration, such as block gages, angle blocks,
length bars etc., are usually slightly different from the nominal dimensions. For instance, a 30 mm
nominal size block gage may have an actual dimension of 30.015 mm. If the deviation of the actual
dimension from the nominal dimension is known, corrections can be made to the measurement
carried out using the standard or an instrument calibrated using the standard. These deviations can
be obtained from calibrations certificates that are supplied with the standards.

Mani Maran Ratnam (2009)

Basic Dimensional Metrology

2.3.4 Error due to ambient condition

Changes in the ambient conditions during measurement from those of international
standard conditions, i.e. temperature of 20C, barometric pressure of 760 mmHg and humidity of
10 mmHg, can cause errors in measurement. Among these factors, temperature has the most
serious effect on the measurement taken. Thus, changes in temperature from the international
standard temperature must be taken into account in most precision measurements. The change in
the dimension measured can be caused by temperature changes in both the part being measured and
the instrument used, and is given by

L = L ( 1t 1 2t 2 )

(2.2)

where,
L is the dimension measured,
t1 is the temperature of part being measured,
t2 is the temperature of measuring instrument,
1 is the thermal expansion coefficient of part material,
2 is the thermal expansion coefficient of instrument material,
t1 = t1 20C
t2 = t2 20C

Contraction (m)

From eq.(2.2) we can see that in order to eliminate error L in measurement both the part
and instrument temperatures must be 20C. In this case, t1 = t2 = 0 and, therefore, L = 0. This
requires the part and the instrument to be kept in a controlled laboratory maintained at 20C.
However, if the temperature is not 20C but is at some arbitrary but equal value, e.g. t1=t2=t, then
L = 0 if 1 = 2. This implies that both the part and instrument must be made from the same
material since they have the same expansion coefficient. In practice, the part and the instrument are
usually made from different materials and their temperatures are difficult to measure. Neither is it
possible to maintain their temperatures at exactly 20C.
Thus, the most effective way of overcoming error due to the temperature changes is to leave
both the part and instrument to achieve stable ambient temperature. Then, the effect of the
deviation of the actual temperature from 20C is taken into account in the evaluating the error. As
an example the effect of temperature on a block gage as a function of time after being handled is
shown in Figure 2.2. The figure shows that although the time has exceeded 30 minutes, the
dimension of the block gage has not returned to its original value.
2.0

0.25

30

15

Time (min.)

Figure 2.2. Cooling curve for a block gage.

10

Mani Maran Ratnam (2009)

Basic Dimensional Metrology

2.3.5 Error due to elastic deformation

When an elastic body is subjected to a force it will undergo deformation. The amount of
deformation will depend mainly on the magnitude of the force and mechanical property of the
material. When an instrument that uses a stylus for measurement is brought into contact with the
part being measured, the pressure due to the stylus will cause the part to deform. This deformation
will introduce an error in the reading.
Consider, for example, a hollow cylinder whose diameter is being measured using a dial
indicator as shown in Figure 2.3. Pressure at the tip of the stylus will cause deformation of the
surface of the cylinder and a small deformation at the stylus itself.

Stylus

Deformation of cylinder
Deformation of stylus

Cylinder

Figure 2.3 Error due to elastic deformation.

The total deformation is given by sum of deformation of cylinder and deformation of
stylus, and is given by the following equation,
1

2
3

= 1.774W (k1 + k

2
2 3

1
1 3
+
R1 R 2

(2.3)

where,
R1 = radius of stylus,
R2 = radius cylinder,
W = pressure at the stylus,
(1 12 )
k1 =
E1
k2 =

(1 )
2
2

E2

where E1 and E2 are the modulus of Young for the stylus and cylinder material and 1 and 2 are the
corresponding Poissons ratios.
Another form of deformation occurs when a body deforms due to its own weight, known as
dead weight. For a beam the error caused by this type of deformation can be minimized by placing
supports at position shown in Figure 2.4. Under this condition, the deflection of the beam, and
hence the error, is minimum.

11

Mani Maran Ratnam (2009)

Basic Dimensional Metrology

Beam

Support
0.577 L

Figure 2.4. Deflection of beam due to dead weight.

2.3.6 Parallax error

Parallax errors occur in the use of instruments like dial gages where there is a gap between
the scale and the pointer and the reading is not taken normal to the scale. An example of parallax
error when reading a dial gage is shown in Figure 2.5(a). Figure 2.5(b) shows an example of parallax
error that can occur when measuring distances using a simple scale rule. Parallax errors can be
overcome by taking the readings normal to the scale or by using an instrument whose pointer
needle is at the same plane as the scale.
A
Direction
reading is
taken

Normal
direction

Pointer

Scale
Reading taken

Parallax
error

Actual reading
(b)
(a)

Figure 2.5. Examples of parallax errors.

2.4 Random errors
The random fluctuation of readings during repeated measurement of the same quantity is
due to an error known as random error. The presence of random errors can only be detected by
repeating the measurement a number of times. Statistical parameters, such as mean and standard
deviation, are used to assess the random errors. For a series of reading x1, x2, x3 ..... xn, the
arithmetic mean x is defined as:
x=

x 1 + x 2 + x 3 + ....... + x n
n

(2.4)

and the standard deviation is given by

12

Mani Maran Ratnam (2009)

Basic Dimensional Metrology

(x x )

(2.5)

n 1

If the measurement of a value x is repeated a large number of times, we will get slightly
different values due to the random errors. A frequency distribution of the measurement can be
obtained by plotting the measured value against the number of times each measurement occurs or
number of readings that fall within a range. The distribution will take the familiar bell-shaped
(Gaussian) profile shown in Figure 2.6 where the variable z is given by
z=

(2.6)

and the Gaussian function is given by

f (x ) =

2
1
e z
2

(2.7)

The function in eq.(2.7) and shown plotted in Figure 2.6 using =2 is also known as the
probability density function. An important property of the curve shown in Figure 2.6 is that 99.8%
of the data lies within 3 of the mean value. For instance, if a sample of 2000 parts, say nails, is
taken and their diameters measured, we can say with 99.8% level of confidence that the diameter
will lie within 3 of the mean diameter, where is the standard deviation of the 2000 samples.
Similarly, 95% of the data lie within 2 of the mean.
f(z)
0.25

0.2

0.15

0.1

0.05

0
-5

-4

-3

-2

-1

Figure 2.6. Gaussion curve for =2.

The uncertainty in measurement caused by random errors must be related to the
manufacturing tolerance of a product. As a rule of thumb, the accuracy of the measuring instrument
must be within 10% of the manufacturing tolerance of the part. For instance, if the manufacturing
tolerance on the diameter of a product is 0.1 mm, then the measuring instrument used to measure
the diameter must have a minimum accuracy of 0.01 mm. If p dan m are the standard deviation
of the manufacturing process and standard deviation of the measurement process respectively, then
the relationship between p dan m will determine number of products that will be rejected or
accepted by mistake at both ends of the tolerance limits. Assume that the nominal diameter is 20.0
mm, then the maximum and minimum diameters acceptable are 20.1 mm and 19.9 mm. At the
13

Mani Maran Ratnam (2009)

Basic Dimensional Metrology

upper limit of 20.1 mm if the measuring instruments shows a reading of 20.12 mm, the product will
be rejected although the reading may be due to the deviation of the measurement process. If the
standard deviation of the measurement process (m) is large compared to the standard deviation of
the manufacturing process (p) more products will be rejected by mistake at the upper limit and
more will be accepted at the lower limit.

Manufacturing process
distribution

Measurement process
distribution

(6m)

Manufacturing tolerance
(6p)

Figure 2.7. Relationship between manufacturing process distribution and measurement

process distribution.

2.5 Compound errors

There are many situations where the measured quantities are used to determine another
variable. For instance the diameter and length of a rod are measured to determine its volume. The
errors in the measurement of diameter and length will contribute to error in the volume. In general,
if a quantity M is a function of several quantities, say a, b, c etc. and the errors in the measurement of
each of these quantities are a, b, c etc., then the error M in M is given by

M =

where

M
M
M
a +
b +
c
a
b
c

M
the partial derivative of M with respect to a
a
M
the partial derivative of M with respect to b and so on.
b

The error M is known as compound error. Exercise 2.3 illustrates a problem where the
compound error is to be determined.

14

Mani Maran Ratnam (2009)

Basic Dimensional Metrology

Revision Exercises
Exercise 2.1
Figure below shows an arrangement for measuring the dimension L on a block using a dial gage. If
the dial gage gives a reading of 5.212 mm, calculate the error in the measurement. Given that angle
= 5.
Dial gage

L
block

Exercise 2.2
The diameter of an aluminium rod is measured using a micrometer in a laboratory where the
temperature is 25C. The reading shown by the micrometer is 12.54 mm. Given that the coefficient
of thermal expansions of aluminium is 2310-6/C and that of the micrometer material is 11.7
10-6/C determine the error in the measurement and the true diameter of the rod.

Exercise 2.3
The volume V of a cylinder is given by the expression: V = r 2 h , where r is the radius and h is the
height of the cylinder. If r = 50 mm and h = 200 mm, and the errors in the measurement of r and h
are, respectively, 0.5 mm and 1.0 mm, determine the error in the volume V.

15

Mani Maran Ratnam (2009)