Quality Assurance and Quality Control

Uncertainty of
Measurement Results
A measurement result is an information. Without
knowledge of its uncertainty it is just a rumour.

Albert Weckenmann
 Overview

Uncertainty definition;
Uncertainty and Error;
Why do we need uncertainty?
When should you evaluate uncertainties of
measurement results?
Uncertainty sources and uncertainty components;
Low of propagation of uncertainty;
Procedures steps for uncertainty evaluation;
Examples;
Spreadsheet method of Kragten for uncertainty
calculation and assessment.
 Uncertainty definition
Basic documents
GUM: Guide to the Expression of Uncertainty in Measurement
(JCGM 100:2008), first published in 1993 by ISO in collaboration
with BIPM, IEC, IFCC, ILAC, IUPAC, IUPAP and OIML;
(freely available from www.bipm.org)

EURACHEM/CITAC-Guide Quantifying Uncertainty in Analytical

Measurement,
3rd edition published in 2012 (QUAM:2012.P1);

(freely available from www.eurachem.org)

EUROPEAN CO-OPERATION for ACCREDITATION Evaluation of

the Uncertainty of Measurement in Calibration (EA-4/02 M: 2013);

(freely available from http://www.european-accreditation.org/)

 Quantity and Uncertainty/Error approach

Quantity - property of a phenomenon, body, or substance, where

the property has a magnitude that can be expressed as a number
and a reference (VIM: 2012; 1.1);

Measurand the quantity intended to be measured;

(VIM: 2012; 2.3);

Error Approach (Traditional Approach or True Value Approach)

to determine an estimate of the true value that is as close as
possible to that single true value. The deviation from the true
value is composed of random and systematic errors.

The objective of measurement in the Uncertainty Approach is not

to determine a true value as closely as possible. Rather, it is
assumed that the information from measurement only permits
assignment of an interval of reasonable values to the
measurand, based on the assumption that no mistakes have
been made in performing the measurement.
 Uncertainty - definition

Non-negative parameter characterizing the

dispersion of the quantity values being attributed to
a measurand, based on the information used
(VIM: 2012, 2.26)

A parameter, associated with the result of a

measurement, that characterizes the dispersion of
the values that could reasonably be attributed to the
measurand
(GUM: 2008, 2.2.3)

result = value uncertainty

 Uncertainty and Error

Ytrue value
YA
Error

measurand
-U +U

Yestimated value YA
of the measurand

Correction for all recognized

systematic effects
 Why do we need uncertainty?

It is an intrinsic part of every measurement result;

It is required by ISO/IEC 17025: 2005 Accreditation;
5.4.6.2 Testing laboratories shall have and shall apply procedures
for estimating uncertainty of measurement;
5.4.6.3 When estimating the uncertainty of measurement, all
uncertainty components which are of importance in the given
situation shall be taken into account using appropriate methods of
analysis.
It improves the knowledge about the measurement
procedure;
It allows comparison of results GUM does not require
statistical tests unless you need it.
 Why do we need uncertainty?

The uncertainty of the result demonstrates the metrological

QUALITY of the measurements (not measuring with the
smallest achievable uncertainty):
In laboratory documents in transparent way the
measurement procedure;
For end-user gives the result with proper confidence;

Measurement uncertainty - is it a criterion for judgement

and placing orders of testing laboratories?
Measurement uncertainties should not be misused as a quality indicator
for laboratories - otherwise there will be a tendency among some
laboratories to report unrealistically low measurement uncertainties;
There is no need to have lower uncertainties then requested from the
customer.
 Why do we need uncertainty?
Customer needs

Before calculating or estimating the measurement

uncertainty, it is recommended to find out what are the
needs of the customers;
The main aim of the actual uncertainty calculations will
be to find out if the laboratory can fulfill the customer
demands;
Customers often are not used to specify demands, so in
many cases the demands have to be set in dialogue with
the customer.
 Why do we need uncertainty?

A well documented uncertainty statement:

Underpins your results and provides transparency;
Identifies major uncertainty contributors -
find out ways to improve the procedure;
Demonstrates compliance with limits (legal or contractual).

Your best defence in discussions!

Repeating the measurement 2, 10 or 100 times gives you

only repeatability or over time within-lab reproducibility
(not total uncertainty)!
 Why do we need uncertainty?
Comparison of results

Mean value
12.5

12.0

11.5
mg kg-1

11.0

10.5
1s U

Lab A Lab B Lab A Lab B Lab A Lab B

 Why do we need uncertainty?
Repeatability / Reproducibility / Precision

Measurement precision: closeness of agreement between

indications or measured quantity values obtained by
replicate measurements on the same or similar objects
under specified conditions (VIM: 2012, 2.15).
Measurement repeatability: measurement precision
under repeatability conditions of measurement
(VIM: 2012, 2.21);
Measurement reproducibility: measurement precision
under reproducibility conditions of measurement
(VIM: 2012, 2.25);
 Why do we need uncertainty?
Compliance against limits

Example: QUAM:2012

A decision rule must be defined!

Guidance can be found in EURACHEM/CITAC (2007)
Use of uncertainty information in compliance assessment.
 When should you evaluate
uncertainties of measurement results?

When a procedure is introduced inside your

laboratory;
When a critical factor changes in the procedure
(instrument, operator, sample type );
During / together with procedure validation.

An individual evaluation is not needed for

every individual result produced!
 When should you evaluate
uncertainties of measurement results?

How often re-evaluation of uncertainty is needed?

In case the laboratory needs to

complains with regulation follows
the respective document;

In case of no guidelines available:

(i) Use a common sense for setting the
frequency;
(ii) Monitor the data from control chart.
 Uncertainty sources

Sampling
Storage conditions
Instrument effects
Reagent purity
Assumed stoichiometry
Measurement conditions
Sample effects
Computation effects
Blank correction
What is usually taken into
Operator effects
account when calculating a
Random effects traditional Confidence
Interval (CI)?
 Uncertainty components
Uncertainty type

Uncertainty components are formally grouped into two

categories based on their method of evaluation, A and B.
Type A evaluation (of uncertainty) - method of evaluation
of uncertainty by the statistical analysis of series of
observations;
Type B evaluation (of uncertainty) - method of evaluation
of uncertainty by means other than the statistical analysis
of series of observations;
(previous experiments, literature data, manufacturers
information, experts estimate).

(GUM: 2008, 0.7, 2.3.2, 2.3.3)

 Uncertainty components
Standard uncertainty - us

When expressed as a standard deviation, an

uncertainty component is known as a standard
uncertainty ( ).
(QUAM:2012; 2.3.1)

Uncertainty of the result of a measurement

expressed as a standard deviation.
(GUM: 2008; 2.3.1)
 Uncertainty components
Combined (standard) uncertainty - uc

For a measurement result y, the total uncertainty, termed

combined standard uncertainty and denoted by uc(y), is an
estimated standard deviation equal to the positive square root
of the total variance obtained by combining all the uncertainty
components, however evaluated, using the law of propagation
of uncertainty or by alternative methods (e.g. the spreadsheet
method of Kragten).
(QUAM:2012; 2.3.2) For example:

In general:
= ( , ,, , ) y
 Uncertainty components
Combined (standard) uncertainty - uc

Law of propagation of uncertainty:

= ( , ,, , )

= . + 2. . . . . ,
,
"

where, = - sensitivity coefficients;

( ), ( ) - standard uncertainties of and ;
- correlation coefficient between and ;
 Uncertainty components
Combined (standard) uncertainty - uc
Law of propagation of uncertainty:
= . + 2. . . . .
= ( , ,, , ) ,
,

"

When all variables

%
, ,, , = .
%
are independent:
When the function involves only sum or difference (e.g.):
= + $ = ( ) + ( ) + ( $)

When the function involves only product or quotient (e.g.):

( ) ( ) ( $)
= = + +
$ $
 Uncertainty components
Expanded uncertainty - U

U is obtained by multiplying the combined standard

uncertainty uc(y), with a coverage factor k;
The expanded uncertainty U provides an interval within
which the value of the measurand is believed to lie with a
higher level of confidence (QUAM:2012; 2.3.3);
The choice of the factor k is based on the level of
confidence desired:
k = 2 corresponded to P = 95%;
k = 3 corresponded to P = 99%;

For normal distribution:

2
3
 (Re)-calculating standard uncertainty (standard deviation)
Confidence Interval (CI)
(),* . + (),* . +
-. -. =
, ,
The used level of confidence () and
degree of freedom (f = N-1) are known.

-.. ,
/ =+=
(),*
 (Re)-calculating standard uncertainty (standard deviation)
Rectangular distribution
p(x) 1
, 2 < <2
0 = 22
0, otherwise
2
/ =+=
x 3
Rectangular distribution is usually described in terms of:
the average value and the range (a);
One can only assume that it is equally probable for the value to lie
anywhere within the interval;
Certificates or other specification give limits where the value could
be, without specifying a level of confidence (or degree of freedom).
Example:
Concentration of calibration standard is quoted as (1000 2) mg/l.
 (Re)-calculating standard uncertainty (standard deviation)
Triangular distribution
2+
, 2 < < 0
p(x) 2
0 = 2
, 0 < < 2
2
0, otherwise

x 2
/ =+=
6
Distribution used when it is suggested that values near the centre of
range are more likely than near to the extremes (Values close to
mathematical expectation are more likely than near the boundaries);
Example:
Volumetric glassware. If any time in doubt, use the
rectangular distribution!
 Procedures steps for uncertainty evaluation
EURACHEM/CITAC Guide

In GUM: 2008 ( 8) is presented summary of procedure for

evaluating and expressing uncertainty;
Approach for uncertainty estimation is also suggested in
QUAM: 2012 (a procedure containing 4 steps):

Step 1

Step 2

Measurand = quantity intended to be measured.

(VIM: 2012; 2.3)
Measurand = the particular kind of quantity to be measured, usually
the concentration or mass fraction of an analyte (QUAM: 2012; 5.2).
 Procedures steps for uncertainty evaluation
EURACHEM/CITAC Guide
Step 2 Express mathematically the relationship
between the measurand y and the input
quantities on which y depends:
Step 3
= ( , ,, , )

Should any correction factor be included?

Input quantities ( ) may be quantities whose

value and standard uncertainty are directly
determined in the current measurement
(evaluation of Type A, statistical analysis of series
of observation) or brought into the measurement
from external sources (evaluation of Type B).

Step 4
Procedures steps for uncertainty evaluation
EURACHEM/CITAC Guide

Step 4

(- ?) mg/g, k = 2
 Procedures steps for uncertainty evaluation
Understanding the measurement

The evaluation of uncertainty is neither a routine task nor a

purely mathematical one; it depends on detailed knowledge
of the nature of the measurand and of the measurement.
(GUM: 2008, 3.4.8)

The state of the art of uncertainty evaluation is determined by

the degree of understanding of measurement, that includes,
critical thinking, professional skills and good understanding of
the processes and chemistry involved!
 Examples
Volume of liquid (solution)

Example 1. Calculate the combined uncertainty of the volume of

water solution prepared in 1000 ml volumetric flask if it is known:
In the certificate of the volumetric flask is declared:
(1000.0 0.2) ml at 20 0C;
The temperature in the laboratory varies within the interval:
(20 5) 0C. The coefficient of volume expansion for water is
2.1x10-4 0C-1 (the volume expansion of the flask itself is neglected);
Repeatability - the uncertainty due to variations in filling is
assessed to be 0.3 ml.

Volumetric flask
certificate
 Examples
Concentration of calibration standard

Example 2. A calibration standard solution of Zn (0.102 mg/l) was

prepared by transferring 1 ml of commercial reference material
(ICP Multi VI - 102 mg/l Zn) into 1000 ml volumetric flask. The flask
was filled to the mark with 0.1 % HNO3 v/v. Calculate the combined
uncertainty of the conc. of the obtained Zn calibration solution, if:
In the certificate of the commercial RM is declared that the
concentration of Zn is (102 5) mg/l no information is supplied
about the degrees of freedom and confidence level;
The standard uncertainty of the pipetted volume is 0.005 ml;
For the standard uncertainty of the volumetric flask volume take
the obtained value from the previous example (Problem 1).

ICP Multi VI CPAchem Muilti-element #2A

certificate certificate
Spreadsheet method of Kragten

Applicable when:
Either
= , ,, , is
linear in ;
or
( ) is small compared to ;
 Method of Kragten
Calculating the uc of solution volume

Example 1.
V flask (ml)= 1000 Variable mean u_s Ser. Temp. Repeat.
Certificate uncertainty (ml)= 0.2 mlSer. 1000 0.08 1000.1 1000.0 1000.0
Temp. variation (0)= 5 mlTemp. 0 0.61 0.0 0.6 0.0
Fill repeatability (ml)= 0.3 mlRepeat. 0 0.30 0.0 0.0 0.3

V flask(ml)= 1000 0.68 1000.08 1000.61 1000.30

diff 0.082 0.606 0.300
diff^2 0.007 0.367 0.090 0.464sum

Contribution, % 1.4% 79.2% 19.4%

1.4%

19.4%
Ser.
Temp.
79.2%
Repeat.
 Method of Kragten
Calculating the uc of standard concentration

Example 2.
C1 (mg/l)= 102 Variable mean u_s C1 V1 pipette V2 flask
C1 certificate uncertainty (mg/l)= 5 mg/lC1 102.0 2.89 104.9 102.0 102.0
V1 pipette (ml)= 1.000 mlV1 pipette 1.000 0.005 1.000 1.005 1.000
V1 u_s (ml)= 0.005 mlV2 flask 1000.0 0.68 1000.0 1000.0 1000.7
V2 flask (ml)= 1000.0
V2 u_s (ml)= 0.68 C2 (mg/l)= 0.102 0.003 1.05E-01 1.03E-01 1.02E-01
diff 2.89E-03 5.10E-04 -6.93E-05
diff^2 8.33E-06 2.60E-07 4.80E-09 8.60E-06sum

Contribution, % 96.9% 3.0% 0.1%

3.0%

C1
V1 pipette
96.9% V2 flask
 Method of Kragten
Calculating the uc of Cx - Bracket calibration

Example 3. Calculating the combined uncertainty of the concentration

of Zn in surface water (real sample determined by FAAS).
mean u_s RSD% Variable mean u_s C3 C4 A3 A4 A
C3 (mg/l)= 1.02 0.0296 2.90 mg/lC3 1.02 0.03 1.05 1.02 1.02 1.02 1.02
C4 (mg/l)= 2.04 0.0594 2.91 mg/lC4 2.04 0.06 2.04 2.10 2.04 2.04 2.04
A3 (signal)= 0.406 0.006 1.48 signalA3 0.406 0.006 0.406 0.406 0.412 0.406 0.406
A4 (signal)= 0.806 0.008 0.99 signalA4 0.806 0.008 0.806 0.806 0.806 0.814 0.806
A (signal)= 0.652 0.008 1.23 signalA 0.652 0.008 0.652 0.652 0.652 0.652 0.660

Cx (mg/l)= 1.65 0.05 1.659 1.684 1.641 1.635 1.668

diff= 0.011 0.037 -0.006 -0.012 0.020
6.3%
diff^2= 0.00013 0.00133 0.00004 0.00015 0.00042 0.00207sum
20.1% C3
7.3% C4 Contribution, % 6.3% 64.5% 1.7% 7.3% 20.1%
A3
64.5%
1.7% A4
Advantages:
A
The uncertainty of the concentration of the
calibration standards is included;
True homoscedastic model;
C3 Cx C4
A3 Ax A4
C 3 ( A4 A X ) + C 4 ( A X A3 )
Cx =
A4 A3
 Uncertainty of Measurement Results
Conclusions

Uncertainty estimation according to the GUM is a

useful and accepted concept:

It allows the analyst to combine prior knowledge and

observations in a consistent and well defined way;

It allows others (e.g. assessors) to understand what and

how things were done;

It doesnt requires to measure with smallest achievable

uncertainty, but with the most realistic one.
 Uncertainty of Measurement Results
Conclusions

Uncertainty adopted and accepted by:

National Metrology Institutes and BIPM;

ISO/IEC 17025: 2005 (required for accreditation);

IUPAC, OIML and accreditation community

such as EA and ILAC;

CEN is incorporating these concepts.

 Uncertainty of Measurement Results

Thank you for your attention!