Expression of

Uncertainty

1
2
Principles of Measurement Uncertainty

3
Expression of Uncertainty

Measurement is just Even with calibration

an estimate because all and doing our good
measurements contain professional practice
limitations in precision we have error in our
error with respect to the measurements relative
"True Value". to the "True Value".

"True Value" is somewhat theoretical.

4
Uncertainty of Measurement

Definition of uncertainty

„A parameter associated with the result of a

measurement, that characterises the dispersion
of the values that could reasonably be attributed
to the measurand.“

(according to the International Vocabulary

of Basic and General Terms in Metrology)

5
Uncertainty of Measurement

Combined standard uncertainty

For a measurement result y, the total uncertainty,

termed combined standard uncertainty and
denoted by uc(y), is an estimated standard
deviation equal to the positive square root of the
total variance obtained by combining all the
uncertainty components, however evaluated,
using the law of propagation of uncertainty.

ucombined = u12 + u22 + ... + un2

6
Uncertainty of Measurement

Expanded uncertainty

For most purposes in analytical chemistry, an

expanded uncertainty U, should be used. The expanded
uncertainty provides an interval within which the value
of the measurand is believed to lie with a higher level
of confidence. U is obtained by multiplying uc(y), the
combined standard uncertainty, by a coverage factor k.
The choice of the factor k is based on the level of
confidence desired. For an approximate level
of 95 % k is 2 and 99% k is 3.

7
Uncertainty of Measurement

Uncertainty of measurement

ucombined = u12 + u22 + ... + un2

Uexpanded = k . ucombined
8
 The Concept of Measurement Uncertainty
7) expanded uncertainty U
statement of confidence that the value lies in the reported range
U: expanded uncertainty
k: coverage factor
U  k  ucombined ( y)
k = 2  level of confidence of 95%
k = 3  level of confidence of 99%

8) report result together with the uncertainty

• ”The value is 20  2 cm. The reported expanded uncertainty is based on
a standard uncertainty multiplied by a coverage factor k = 2, providing a
level of confidence of approximately 95%.”
• It should be stated how the uncertainty was obtained and what it includes.
• Two significant digits should be given.

9
Uncertainty of Measurement

Error

It is important to distinguish between

error and uncertainty. Error is defined as the
difference between an individual result and the
true value of the measurand. As such, error is
a single value. In principle, the value of a known
error can be applied as a correction to the result.

10
 290
295
300
305
310
315
Ref.

10
Error

11

12

13

14

15

16
Uncertainty

17

18

19

20

Mean
11
 3,5%

3,0%
Preciso, pero Ni correcto y
no correcto ni preciso

2,5%

Error > Uncertainty

Error %

2,0%

1,5%

Uncertainty > Error

1,0%

0,5%
Correcto, pero
Correcto y preciso no preciso

Uncertainty %
0,0%
0,0% 0,5% 1,0% 1,5% 2,0% 2,5% 3,0% 3,5%

12
Uncertainty of Measurement

Objectives
Reasonable estimation of uncertainty by
identification of all components of uncertainty.

Ensurance that the results in the test reports are not

giving a wrong impression about the uncertainty.

Required because
Required by
of the existance of narrow
the test
limits on which decicious
method
on conformance to a
Required by
specification are based
the client

13
Uncertainty of Measurement

Check: Existance of normative requirements

1. on the uncertainty of measurement?
yes Regulations in the
special working instruction
no
Check: „Conflict“ of test results with limits of
2. specifications because of uncertainty?
yes
no
Check: Does the client require the estimation
3. of uncertainty?
no No estimation of
uncertainty
yes
14
Uncertainty of Measurement

Identification of all components

4. of uncertainty

Calculation and / or estimation of

5. the uncertainty components

Evaluation of the uncertainty contributions

6. to the combined uncertainty

Calculation of the combined /

7. Expanded uncertainty

15
Uncertainty of Measurement

Estimation Calculation

Fulfilment of the client‘s needs

16
Uncertainty of Measurement

Random Sampling
Storage
effects
conditions
Operator
effects
Instrument
effects
Blank Sources of
correction uncertainty
Reagent
Computational purity
effects

Sample Assumed
effects Measurement stoichiometry
conditions
17
 The standard uncertainty quantifies the extent to
which our knowlegde of the "Measurement Error"
is spread or smeared relative to the "True Value".

Statistical tests

Measurement "Measurement
risks
Uncertainty"
Other decision-
related activities
is often
Standard Uncertainty Measurement Uncertainty
referred to
18
Uncertainty of Measurement

Uncertainty of measurement

ucombined = s12 + s22 + ... + sn2

Uexpanded = k . ucombined
19
 Quantifying the single uncertainty components:
Type A

1 n x1  x2  x3  ...  xn x
x  i
n i 1
x 
n
Arithmetic mean of
n (independent) observations

1

n
s ( xi )  ( xi  x ) 2 experimental standard deviation
(n  1) i 1

s ( xi ) 1
 experimental standard deviation
n
s( x )   ( x  x ) 2

n(n  1) i 1
i
n of the mean

u ( xi )  s ( x ) for several series u(xi): the standard uncertainty

u ( xi )  s ( x) for one series of measurement

20
 Uncertainty in Microbiology
20 measurements of
the same sample Example: Total bacterial
count in hamburger
Results Log10
1.1 x 104 4.04
1.1 x 104 4.04
1.1 x 104 4.04 Mean 4.4 x 104 cfu‘s/g
1.1 x 104 4.04
1.1 x 104 4.04
3.0 x 105 4.47 SD 0.52 x 104 cfu‘s/g
4.8 x 104 4.68
2.7 x 105 5.43
7.8 x 103 3.89 0.52  10 4
6.7 x 104 4.83 Relative SD  100%  12%
1.9 x 104 4.28 4.4  10 4

8.0 x 103 3.90

2.2 x 104 4.34
1.2 x 105 5.10 Expanded
Uexpanded = ± 24%
1.2 x 104 4.10 uncertainty
1.5 x 105 5.18
1.7 x 104 4.23
3.0 x 104 4.48 4.4 ± 1.1 x 104 cfu‘s/g
0.5 x 104 3.70 Result
2.0 x 104 4.30 4.4 x 104 ± 24% cfu‘s/g
21
 The Concept of Measurement Uncertainty
6) determine the combined standard uncertainty uc
from the single uncertainty components
first order Taylor series approximation, “law of propagation of uncertainty”
2
n
 f  2
ucombined ( y )     u ( xi )
2
for non-correlated uncertainty components
i 1  xi 

n n
 f  f 
ucombined ( y )     u ( xi x j )
2 for correlated uncertainty components

i 1 j 1  xi  x j


for simple equations:

n
ucombined ( y)  ( xi )  u1  u2  ...  un
2 2 2 2
u
y  x1  x2  x3  x4 ... i 1

2 2 2
y  x1  x2  x3 / x4 ... ucombined ( y) u 2 ( xi ) u1 u2 un
i1 x 2
n
  2
  ...  2
y i x1
x2 2 xn

22
 Laws
Standards
Accepted
statements in
literature

Results of
proficiency tests

Quality control charts

and data from validation

Quantifying according Experience

to EURACHEM GUIDE of experts
23
Uncertainty of Measurement, standardized methods

Standard Methods

Uncertainties under
Defined uncertainties
control, e.g. control charts

Main sources of
uncertainties are known

24
 Measurement uncertainty determined by
interlaboratory comparisons

4. Countrywide interlaboratory comparison on sulfate

in waste water using Ion Chromotography

121 participants gave results mean value

Standard deviation
s= 4%
of the mean value

s = u = ucombined =  4 %

(k = 2) Uexpanded =  8 %
25
 Individual Control Chart
Date mg/l SO42- Ionchromatography
02.05.00 1,00
29.04.00 1,06
05.05.00 0,94
08.05.00 1,00
08.05.00 0,97
09.05.00 0,94
09.05.00 1,00
10.05.00 1,06
10.05.00 1,00
11.05.00 1,03
11.05.00 0,94
12.05.00 1,00
12.05.00 0,97
12.05.00 1,00
15.05.00 1,03
15.05.00 0,97
16.05.00 1,03
16.05.00 1,00
17.05.00 1,03
17.05.00 0,97 SO4 mg/l - 3s 0,881
19.05.00 1,00
19.05.00 1,00
mean 0,994 - 2s 0,919
22.05.00 1,06 s 0,038 + 2s 1,069
22.05.00 1,06 VK % 3,774 + 3s 1,107

26
 Measurement uncertainty determined by
mean value control charts

Task Determination of naphthaline with HPLC

Mean value from 23 measurements c = 0,78 µg / l

Standard deviation (68%-level) 0,056 µg / l

Without further process steps u=s

ucombined = 0,056 µg / l  7,2 %

Uexpanded = 2 x 0,56 µg / l = 0,11 µg / l (k = 2)  14,4 %

Result c = 0,78  0,11 µg / l

27
 Action on estimation of
measurement uncertainty of analysis
medium A B C D1 D2
sampling preparation of sample preparation disturbance of precision of the expanded
the laboratory (distillation, the solution to measurement uncertainty U
sample clearing, be measured procedure with k=2 [%]
(homogenization, extraction, (remaining matrix, (incl. calibration) WITHOUT
reductio)n derivatisation) cross sensitivity) D2

slight until
moderate volatile
compounds

water 1,5 2,0 1,0 5,4

waste water 3,0 2,0 4,0 10,8

soil 10,0 2,0 4,0 21,9

recycling materials 15,0 2,0 8,0 34,2

extremely contami-
nated sites 20,0 2,0 15,0 50,2
heavily until not
volatile
compounds

water 1,0 4,0 1,0 8,5

waste water 2,0 4,0 4,0 12,0
soil 7,0 4,0 4,0 18,0
recycling materials 10,0 4,0 8,0 26,8
extremely contami-
nated sites 15,0 4,0 15,0 43,2
28
 Code: QMF504-2
Central Lab. Quality Management Form Revision: 01
Date: ...
Measurement Uncertainty Page: 1 of 1

QMF504-2 Measurement uncertainty of selected test procedures

Range of Measurement Method of

No./Code Test procedure Matrix
concentration uncertainty estimation
water, +/- 0.3 % acc. to GUM;
1 pH value 1 - 14 +/- 0.4 % mean value control
food
charts

chloride +/- 1 % acc. to GUM;

2 20 – 1000 mg/l water +/- 1.2 % mean value control
(tri-metric)
charts

3 pesticides dependent on +/- 60 % literature

the substance food

4 Pb, Cd 0.01 – 0.1 mg/kg food +/- 20 % proficiency test

Compiled: Checked: Cleared: Valid from:

[Date, Signature] [Date, Signature] [Date, Signature] [Date]

29
Determination of Cadmium in Food Stuff

1 Introduction (Step 1, Specify measurand)

After homogenization and mineralization the Cadmium

content is analyzed by means of AAS.

2 Principle and working range of the test method

With this method amounts of 0.5 – 5 mg Cadmium per

kg food stuff can be analyzed. After homogenization and
mineralization with nitric acid the sample is analyzed by
means of AAS using the graphite furnace technique.

30
3 Method description

0.5 g ± 0.1 mg food stuff is taken for mineralization using

2.5 ml nitric acid. After mineralization the solution is
transferred to a 100 ml flask quantitatively and analyzed
by AAS using graphite furnace technique and calibration
with commercially available standards.

31
 Calculation of measurement uncertainty acc. to GUM

Preparation of the sample

Weighing

Preparation of calibration
Mineralisation standard

AAS Calibration of AAS

Result

32
 Identifying the uncertainty sources
”cause and effect diagrams”
mP
cCd 
V
Volume Purity

Temperature

Calibration

Repeatability
Concentration
cCd
Repeatability Repeatability
m (gross)

Linearity Linearity
Repeatability Repeatability

Sensitivity Sensitivity

Calibration Calibration

Mass

33
5 Identify and analysing uncertainty sources

 Uncertainty of weighing the sample (uW)

 Uncertainty of volume (uV)

 Uncertainty of stock solution for calibration (uCstock)

 Uncertainty of calibration function

34
5.1 Uncertainty of weighing the sample (uW)

The calibration certificate of the balance quotes

± 0.15 mg for the linearity.
0.15 mg
u (W)   0.09 mg
3
This contribution has to be counted twice, once for
the tare and once for the gross weight, because each
is an independent observation and the linearity
effects are not correlable.

This gives a standard uncertainty of

u (W)  2  (0.09) 2 u (W)  0.13 mg

35
5.2 Uncertainty of the volume (uV)

0.5 g are transferred after digestion to 100 ml flask

• Volume effect: The certificate of the supplier of the

100 ml flask quotes ± 0.1 ml for the uncertainty.

0.1 ml
u (V) 
3
u (V)  0.06 ml

36
• Parallax effect: Performed 10 times,
standard deviation: 0.05

100.2
0.05
100.0
100.0
u (P) 
99.9
10
:
: u (P)  0.02 ml
100.0

s = 0.05
n = 10

37
• Temperature effect: room temperature 20 °C ± 3 °C

Coefficient of volume expansion for water 2.1 x 10-4 / °C

4
100 ml  2.110  3 C
u (T) 
3C
u (T)  0.035 ml

38
Standard uncertainty of 100 ml flask

s (V100ml )  u (V)  u (P)  u (T)

2 2 2

s(V100ml)  0.06  0.02  0.035

2 2 2

s (V100ml )  0.07 ml

39
5.3 Uncertainty of Calibration standard (uCstock)

Certificate of supplier 1000 ± 5 mg/l

5 mg
u (Cstock ) 
3 l
u (Cstock )  2.89 mg/l

40
Correlation coefficient r = 0.9997

Standard deviation of the calibration: 0.0040

a = 0.00017
b = 0.0865 slope
S 1 1 (c0  c) 2 n
u(c 0 )    Sxx   (ci  c) 2
B1 p n Sxx i 1
S : residual standard deviation
B1 : slope
p : number of measurements to determine c0
n : number of measurementsfor the calibration
c0 : determined cadmium concentration
c : mean value of the different calibration standards
Sxx :
i : index for the number of calibration standards
j : index for the number of measurements to obtain
the calibration curve
41
5.4 Uncertainty arising from calibration

Standard
1 2 3
(ppb)
0.5 0.058 0.057 0.061
1 0.104 0.104 0.104
2 0.197 0.194 0.194
3 0.284 0.279 0.285
4 0.363 0.349 0.367
Sample 0.07 0.068 0.069

42
 Standard
y x(ci) (ci-cort) (ci-cort)2
(ppb)
0.5 0.058 0.47 -1.83 3.33
0.5 0.057 0.46 -1.84 3.38
0.5 0.061 0.51 -1.79 3.21
1.0 0.104 1.01 -1.29 1.67
1.0 0.104 1.01 -1.29 1.67
1.0 0.104 1.01 -1.29 1.67
2.0 0.197 2.08 -0.22 0.05
2.0 0.194 2.05 -0.25 0.06
2.0 0.194 2.05 -0.25 0.06
3.0 0.284 3.09 0.79 0.62
3.0 0.279 0.03 0.73 0.53
3.0 0.285 3.10 0.80 0.64
4.0 0.363 4.00 1.70 2.89
4.0 0.349 3.84 1.54 2.37
4.0 0.367 4.05 1.75 3.05

2.30 Sxx 25.22

Result of sample: 0.6 µg/l
43
 n
Sxx   (ci  c)  25.22
2

i 1

0.0040 1 1 (0.60  2.3) 2

u(c 0 )     0.033 µg/l
0.0865 3 15 25.22

44
6 Combined uncertainty

2
 u W   u V   u Cstock   u C0 
2 2 2
u C (Cd)
         
c(Cd)  C W   C V   CCstock   CC0 

2 2 2 2
u C (Cd)  0.13   0.07   2.89   0.033 
        
c(Cd)  0.5   100   1000   0.6 

u C (Cd)
 0.0262  0.0007 2  0.002892  0.0552
c(Cd)

u C (Cd)
 0.06
c(Cd)

45
 c0  V
m
W
0.6  0.1
m  0.12 mg/kg
0.5

U c (Cd)  0.06  0.12  0.007  0.01

Cd  0.12  0.01 mg/kg

46
7 Expanded Uncertainty

k = coverage factor
k = 2 (95 % confidence interval)

8 Reporting the Result

Cd = 0.12 ± 0.02 mg/kg

(± expanded uncertainty, coverage factor = 2)

47
 t s
U
n
t = Student-t-factor

s = standard deviation at the 68 %

confidence level

n = number of tests

48
 Practical example:
Estimation of the uncertainty of the measurement
of 30 mg Sulfate in waste water with ion-chromatography

A Result of interlaboratory comparison:

• The laboratory has participated successfully in the 4th all-German
waste-water proficiency testing.
• The standard deviation of the laboratories‘ results was
s = 4%
• Step 6): s = u = ucombined = 4 %  ucombined = 1,2 mg/l für 30 mg/l
• Step 7): Uexpanded= k·ucombined = 2·1,2 mg/l = 2,4 mg/l
• Step 8): Sulfate: 30 mg/l (measurement uncertainty: 2,4 mg/)
This uncertainty is an expanded uncertainty and was obtained by
multiplying the combined uncertainty with the factor k = 2, relating to
a level of confidence of 95%.

49
 Practical example:
Estimation of the uncertainty of the measurement
of 30 mg Sulfate in waste water with ion-chromatography

B Validation data in the method standard

• The validation data:sulfate in waste water, ionchromatography,
ISO 10304-2 (D20): reproducibility standard deviations:
s = 3-6%
• Step 6): s = u = ucombined 
ucombined = between 0,9 and 1,8 mg/l for 30 mg/l
• Step 7): Uexpanded= k·ucombined 
Uexpanded = between 1,8 and 3,6 mg/l for 30 mg/l
• Step 8): Sulfate: 30 mg/l (measurement uncertainty: 1,8-3,6 mg)

50
 Practical example:
Estimation of the uncertainty of the measurement
of 30 mg Sulfate in waste water with ion-chromatography

C Control Charts
• The laboratory maintains an individual control chart for determina-
tion of sulfate. The standard deviation is: s = 3,8%
• control sample measured directly  sample 30 mg/l is diluted.
dilution step carried out 10 times, standard deviation: sdilute = 1%
• Step 6): ucombined = uMKK  u Dil  3,82  12  3,9%
2 2

ucombined = 1,2 mg/l for 30 mg/l

• Step 7): Uexpanded= k·ucombined =2·1,2 mg/l = 2,4 mg/l
• Step 8): Sulfate: 30 mg/l (measurement uncertainty: 2,4 mg)

51
 Practical example:
Estimation of the uncertainty of the measurement
of 30 mg Sulfate in waste water with ion-chromatography

Uncertainty Source Uncertainty budget [mg/l]

Interlaboratory comparisons 2,4
Method validation data 1,8-3,6
Control Charts 2,4

52
 Standards and guidelines
 ”Guide to the Expression of Uncertainty im Measurement” (GUM),
1993, ISO (ca. 70 EUR)
 FD X 07-021 French Standard ”Aid in the procedure for estimating and using
uncertainty in measurements and test results”, 1999, AFNOR
 ISO 5725-1-6:1994Accuracy (trueness and precision) of measurement methods
and results
 ”Quantifying Uncertainty in Analytical Measurement”, EURACHEM / CITAC
Guide, 1995 / 2000, free of costs: www.eurachem.bam.de
 ”Expression of the Uncertainty of Measurement in Calibration”,
EA-4/02, 1999
 ”The Expression of Uncertainty and Confidence in Measurement”, NAMAS NIS
3003, 1997, NAMAS, United Kingdom Accreditation Service
 ”Guidelines for evaluating and expressing uncertainty of NIST measurement
results”, B. N. Taylor and C. E. Kuyatt, NIST, 1993
 ”Measurement Uncertainy in Testing – a collection for beginners”, EUROLAB
Technical Report to be published

53
Estimated Measurement Uncertainties
from Food Industry

• Listeria monocytogenes: ± 0.20 log CFU/ml

• Aerobic mesophilic counts: ± 0.15 log CFU/ml
• Bacillus cereus: ± 0.20 log CFU/ml
• Coliforms: ± 0.25 log CFU/ml
• Enterobacteriacea: ± 0.25 log CFU/ml
• Staphylococcus aureus: ± 0.20 log CFU/ml
• Yeast: ± 0.20 log CFU/ml
• Aerobic mesophilic spores: ± 0.25 log CFU/ml
• Clostridium perfringens: ± 0.20 log CFU/ml

54
55
Upper limit

The upper limit is exceeded with 95 % propability if the

difference between the measured value and the upper
limit is bigger than the critical difference.
56
 Critical Difference
(Directive 213/2001 EC from 09.01.2001)

Evaluation of test results in

Use view of possible limit-
exceeding

The measurement uncertainty is

Starting-point
expressed as critical difference CrD

Knowledge of repeatability and

Condition reproduceability standard deviation,
confidence level 95 % for r and R
57
 The critical difference (95 % confidence level)
can be calculated:

(n  1)
u ( y )  s y  s R  sr2 2

(n  1)
CrD95 ( y  m )  0,59 R  r 2 2

n
58
 Uncertainty in Microbiology
Example: Skimmed
milk powder: Max. 40,000 cfu‘s/g

Proposed uncertainties

Repeatability r = 0.43 (log-units)

Reproducebility R = 0.85 (log-units)

Critical difference CrD = 0.47 (log-units)

78,000 cfu/g (double determination)

Conclusion: one result between 40,000 cfu‘s/g

and 118,000 cfu‘s/g in a series of 5 results
is acceptable.
59
The End

60