REFERENCE MATERIALS IN QUALITY ASSURANCE

OF TESTING LABORATORIES

Jukka Häyrynen
Heikki Kallio
Anna-Maija Kosonen
 NORDTEST
Tasks
The tasks of Nordtest are to promote the safety of life, health, environment and material
values and to encourage a free exchange to trade. The approach adopted by Nordtest to
achieve its objectives is:

- to develop, adopt and recommend test methods and to promote the use of these
by industry and the authorities and also in the standardisation work

- to obtain international recognition of test results and also the competence of the
Nordic countries, for instance by quality assurance and verification of testing
activity

- to endeavour that tests and approval of test results are made in a resource and
cost effective manner

- to promote the technical testing infrastructure in the Nordic countries by means

of research, development of competence and collaboratian, and

- to participate in the European and international development of testing and to

promote Nordic interests,

Organisation
The organisation consists of a board, a secretariat and nine technical groups. These groups
are Acoustics and Noise, Building, Electronics, Environment, Fire, Mechanics, Polymers, VVS
(Mechanical Building Services) and Quality Assurance.

The work is directed by the board which comprises representatives of all the Nordic
countries, The members are appointed by the government or appropriate department of the
country concerned.

The technical groups initiate and evaluate projects. The projects are often structured in such
a way that they can be used as catalysist for development of the combined technical
competence in the Nordic countries. At present, about 250 Nordic projects are being carried
out in some 40 firms and institutions.

The board as well as the technicai groups are assisted by the secretariat which is responsible
for day to day activity. The secretariat is located at Esbo, Finland.

Financial framework
The cost of the Nordtest secretariat and a Iarge proportion of project activity is financed
from the budget of the Nordic Council of Ministers. The grant for 1992 is approx. 2 millions
ECU, The work of the board and the technical groups is financed by the participating
organisations.

Publications
- Register of 1300 test methods and technical reports
- Test methods
- Technical reports
 NT TECHN REPORT 177

Approved 1992-04

Authors: NORDTEST project number: 956-90-l

Jukka Häyrynen
Heikki Kallio Institution: VTT
Anna-Maija Kosonen

Title (English): -

Title (Original): REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTING

LABORATORIES
Abstract:

This paper is the final report of NORDTEST Project 956-90-l “Referensmaterial vid

kvalitetssäkring av provningslaboratorier” (Reference materials in quality assurance of testing

laboratories). The scope of this project was to collect information for the users of reference

materials in the Nordic countries and to propose recommendations for the use of reference

materials

Technical Group: Quality Assurance

ISSN: 0283-7234 Language: English Pages: 35

Class (UDC): 620.1 Key words: reference materials, quality assurance, testing
laboratories

Distributed by: Publication code:

NORDTEST
Tekniikantie 12
FIN-02150 Espoo
Finland
CONTENTS

1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Certification of reference materials . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Organizations within the CRM schemes . . . . . . . . . . . . . . . . . . 7

2 REFERENCES IN QUALITY AND TESTING STANDARDS . . . . . . . . . . . . . . 8

2.1 EN45001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 ISO/IEC Guide 25 : 1990(E) . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.3 IS0 9001 : 1987 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.4 NAMAS accreditation standard Ml0 . . . . . . . . . . . . . . . . . . . . . 10
2.4.1 NAMAS information sheets . . . . . . . . . . . . . . . . 10
2.5 Testing standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5.1 Hardness test standards . . . . . . . . . . . . . . . . . . . 11
2.5.2 Impact test standards . . . . . . . . . . . . . . . . . . . . . 12

3 CHOOSING AND MAINTAINING REFERENCE MATERIALS . . . . . . . . . . . . 13

3.1 Choosing a reference material . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Working standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3 Household practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4 REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTING

LABORATORIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.1 Indirect verification of test instruments . . . . . . . . . . . . . . . . . . . 16
4.1.1 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.2 Controlling laboratory performance . . . . . . . . . . . . . . . . . . . . . . 17
4.2.1 X-R charts . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.2.2 Value-time charts . . . . . . . . . . . . . . . . . . . . . . . 19
4.3 Using the control charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.4 Interlaboratory tests in estimating laboratory performance
and the uncertainty of testing . . . . . . . . . . . . . . . . . . . 21

5 SOME SOURCES OF CERTIFIED MATERIALS . . . . . . . . . . . . . . . . . . . . . 22

5.1 BCR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.2 NIST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

6 COMAR DATABASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

7 NORDIC QUESTIONNAIRE ON REFERENCE MATERIALS . . . . . . . . . . . 24

8 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

APPENDIX 1 Excerpts from BCR REFERENCE MATERIALS CATALOG 2p.

APPENDIX 2 Example of a certificate of measurement 2p.
APPENDIX 3 Producers of RMs and CRMs 1p.
 -3-

1 INTRODUCTION

High-quality quantitative information on the properties of materials is needed for

the safe, efficient and reliable design and use of modern industrial products.
Confidence in the materials and the material properties is being enhanced through
standardized test methods and verified performance of laboratories; formal
accreditation is chosen as an accepted path in Europe towards harmonized
practices and results. Accredited laboratories are expected to perform tests
consistently from day to day; furthermore a test carried out at different accredited
laboratories according to a standardized method should produce identical results.

Reference materials (RMs) are used in various fields of testing to provide users
with accurate reference values and traceability for their measurements, to quantify
and to calibrate methods and equipment and to monitor the performance of the
laboratory. These tests can be complex, or the result is qualitative or is not readily
traceable to national or international standards. A reference material is by defini-
tion a material or substance one or more properties of which are sufficiently well
established to be used for calibration of an apparatus, assessment of a measure-
ment method, or for assigning values to materials1.

RMs are widely used within the field of analytical chemistry, where they form the
basis for ensuring accuracy and for adhering to valid measurement systems. The
use of reference materials is gaining momentum in other fields of testing, too.

Certified reference materials (CRMs) are reference materials whose property

values are certified by a technically valid procedure, accompanied by or traceable
to a certificate or other documentation which is issued by a certifying body1. In
chemical laboratories CRMs correspond to the reference standards of physical
measurements. The use of CRMs has been encouraged and emphasized by the

1
definition, ISO GUIDE 35
 -4-

calibration and traceability requirements in the international standards also for

accreditation of testing laboratories.

1.1 Definitions

There are a number of terms often used when dealing with quality assurance of
testing as well as with calibration of equipment and instruments. The definitions
below are collected from various standards to clarify further discussions in the
paper.

REFERENCE MATERIAL & CERTIFIED REFERENCE MATERIAL

* see above

WORKING STANDARD
* Reasonably homogeneous, previously analyzed materials, pure com-
pounds, solutions of pure elements etc. used instead of CRMs for fre-
quent checks and in situations when cost or availability limit the use of
CRMs.2

STANDARD REFERENCE MATERIAL (NIST Standard Reference Materials

Catalogue 1990-91, NIST Special Publication 260)
* NIST Standard Reference Material (SRM) is a certified reference
material issued by NIST.

2
Distinction between RMs and working standards is somewhat vague; one
way could be: RM is commercially available whereas a working standard
is manufactured in-house.
 -5-

REFERENCE STANDARD OF MEASUREMENT (NAMAS ACCREDIT

ATION STANDARD M10)
* Instrument or gauge used to calibrate measuring and test equipment.
The term also applies to a material used as a reference standard.

Note: The term “standard reference material” is also used in NAMAS

standards for a certified reference material.

CERTIFIED VALUE (ISO-GUIDE 30)

* For a CRM, the value that appears in the certificate or other documen-
tation accompanying the material, this value having been certified by a
technically valid procedure.

CONSENSUS VALUE (ISO-GUIDE 30)

* For a reference material, the value of the quantity obtained by
interlaboratory testing, or by agreement between appropriate bodies or
experts.

CALIBRATION (ISO GUIDE 25 : 1990)

* The set of operations which establish, under specified conditions, the
relationship between values indicated by a measuring instrument or
measuring system, or values represented by a material measure, and the
corresponding known values of a measurand.

VERIFICATION (ISO GUIDE 25 : 1990)

* Confirmation by examination and provision of evidence that specified
requirements have been met.

* NOTE - In connection with the management of measuring

equipment, verification provides a means for checking
that the deviations between values indicated by a mea-
 -6-

suring instrument and corresponding known values of a

measured quantity are consistently smaller than the
maximum allowable error defined in a standard, regula-
tion or specification peculiar to the management of the
measuring equipment.

* The result of verification leads to a decision either to restore to service,

or to perform adjustments, or to repair, or to downgrade, or to declare
obsolete. In all cases it is required that a written trace of the verification
performed be kept on the measuring instrument’s individual record.

DIRECT VERIFICATION OF TESTING EQUIPMENT (ISO / EN standards)

* Method for checking the main functions of the machine.
* NOTE - This method shall be used:
When the machine is installed or removed or repaired.
When the result of the indirect verification is not satisfactory.

INDIRECT VERIFICATION OF TESTING EQUIPMENT (ISO / EN stan-

dards)
* Method suitable for overall checking of the machine. The indirect
verification method may be used on its own for periodic routine checking
of the machine in service.

* The indirect verification may be carried out by means of certified

reference materials (“standardized, calibrated blocks”).

VERIFICATION OF A REFERENCE MATERIAL (ISO standards)

* All the operations carried out, normally by a technically competent
third party, with the object of confirming the validity of the value(s)
stated in a reference material certificate.
 -7-

CALIBRATION OF STANDARDIZED BLOCKS (EN standards)

* A specified method to be used to gain required traceability for the
reference material used for the indirect verification of the testing
machines.

1.2 Certification of reference materials

Reference materials can be gaseous, liquid or solid. Certification of CRMs can be

done according to three approaches:

DEFINITIVE METHOD, certification is done by one highly qualified laboratory

using the best possible methods to reach very high precision and very
low systematic error.

INDEPENDENT METHOD, usually done by one laboratory using two or more

reliable independent methods of measurement, each of which has been
shown to yield good results. The methods produce results which overlap
and all results by all the methods must fall within allowable end-user
uncertainty.

INTERLABORATORY METHOD, a number (5 - 20) of laboratories carry out

an interlaboratory test programme, and a statistical analysis of the results
is carried out. The programmes often consist of two or more rounds of
measurements to cut down the uncertainties due to different methods of
handling, measurement and analysis of the samples.

1.3 Organizations within the CRM schemes

In Europe the Community Bureau of Reference (BCR) is the largest establishment

dealing with certified reference materials (see appendix 1). There are also other
 -8-

establishments with more limited fields of interest. In the US the National

Institute of Standards & Technology is federally responsible for the CRMs.

The ISO Committee on reference materials (REMCO) is harmonizing the work on

reference materials by organizing relevant research projects and by publishing ISO
GUIDES on the subject. REMCO has prepared an international database on
reference materials (COMAR, see chapter 6).

2 REFERENCES IN QUALITY AND TESTING STANDARDS

The quality-related standards either recommend or require the use of reference

materials to increase the reliability of testing activity. The following excerpts from
a number of standards show the level of requirements.

2.1 EN 45001

The EN 45001 standard “General criteria for the operation of testing laboratories”
describing the requirements for the accreditation of laboratories refers to reference
materials as follows:

* Reference materials shall where possible be traceable to national or

international standard reference materials. (5.3.3. Equipment)

* The Quality Manual shall contain at least:

...
f) where appropriate, reference to proficiency testing, use of
reference materials, etc.; (5.4.2. Quality System)
 -9-

2.2 ISO/IEC Guide 25 : 1990(E)

The ISO/IEC Guide 25 “General requirements for the competence of calibration

and testing laboratories” describes the activities of both calibration and testing
laboratories as follows:

* The quality manual, and related quality documentation, shall state the
laboratory’s policies and operational procedures established in order to
meet the requirements of this Guide, the quality manual and related
quality documentation shall also contain
...
n) reference to verification practices including interlaboratory
comparisons, proficiency testing programmes, use of reference
materials and internal quality control schemes. (5.2, Quality
system, audit and review)

* Reference materials shall, where possible, be traceable to national or

international standards of measurements, or to national or international
standard reference materials. (9.7, Measurement traceability and calibra-
tion)

2.3 ISO 9001 : 1987

The ISO 9001 standard “Quality systems. Model for quality assurance in design/
development, production, installation and servicing” describes the requirements for
inspection, measuring and test equipment as follows:

* The supplier shall control, calibrate and maintain inspection, measuring

and test equipment, whether owned by the supplier, on loan, or provided
by the purchaser, to demonstrate the conformance of product to the
specified requirements. Equipment shall be used in a manner which
 -10-

ensures that measurement uncertainty is known and is consistent with the

required measurement capability. (4.11)

2.4 NAMAS accreditation standard Ml0

The NAMAS accreditation standard “General Criteria of Competence for Calibra-

tion and Testing Laboratories” describes the requirements for measurement
traceability and calibration as follows:

* Where the concept of traceability is not in practice, the Laboratory

shall provide satisfactory evidence of correlation of calibration or test
procedures, for example by participation in a suitable program of mea-
surement audit or inter-laboratory comparison or through the regular use
of standard reference materials. (7.5)

* Where the reference standard of measurement is a reference material

the Laboratory shall, wherever possible, obtain a certificate that provides
evidence of the characterization of the material and evidence of traceabil-
ity to national or international standards of measurement, or national or
international reference materials. (7.7)

2.4.1 NAMAS information sheets

NAMAS has published various information sheets as supportive media for

laboratories applying for or maintaining accreditation. These information sheets
give requirements, for example, for test performance, machine verification, use of
reference materials and also give sources of accredited verification and test
blocks. (NAMAS Information Sheets NIS 13 ‘TRACEABILITY, HARDNESS
MEASUREMENTS”)
 -11-

2.5 Testing standards

Many testing standards referring to calibration or traceability do require reference

materials either as primary or secondary means of traceability. Examples of such
methods are measurement of surface hardness and impact testing of metallic
materials. These are chosen below as examples of test methods.

ISO or EN standards on mechanical testing usually have separate standards for the
verification of testing machines. For example hardness and impact testing
machines shall or may be verified by means of standardized blocks (=RMs). This
method is so-called indirect verification (see definitions).

2.5.1 Hardness test standards

ISO/EN
New EN standards will incorporate corresponding ISO standards and replace the
existing European national standards. These standards contain both direct and
indirect verification methods of testing machines. There are also separate stan-
dards for calibration of standardized blocks to be used for indirect verification of
hardness testing machines.

Direct verification (see definitions) of testing machines involves

-verification of test force,
-verification of the indenter and
-verification of the measuring device.

The hardness values given by the testing machine do not depend only on the
dimensions of the indenter, but also on the surface roughness and the seating of
the indenter. For this reason an indirect verification using RMs is considered
necessary. The indirect verification can be performed with two procedures.
According to the first procedure each hardness scale (Rockwell), different
 -12-

hardness ranges and test forces or ball sizes (Brinell) will be verified periodically.
The other possibility is to verify prior to the test only one hardness value that
corresponds approximately to that of the tests to be performed.

ASTM
In ASTM standards instructions for a specific hardness testing method, verifica-
tion of machine and calibration of hardness blocks are in one standard. According
to ASTM standards the direct verification shall be used for new and rebuilt
machines. The indirect method using standardized test blocks may be used in
routine testing to assure the operator that a machine is operating properly.

2.5.2 Impact test standards

According to prEN 10 045 Part 2, the main method for verification of impact test
machines is the indirect verification using the Charpy V reference test pieces,
whose breaking energies are known. The dimensional requirements are more
accurate than the requirement for standard test specimens. The test pieces shall be
national test pieces traceable to the BCR test pieces or certified BCR test pieces.
The direct method shall be used, when the machine is being installed or repaired
or if the indirect method gives an incorrect or unacceptable result.

Contrary to the EN standard, the main method in the ISO 442 standard is the
direct verification. The indirect method is not even an alternative to complete
direct verification. The reference test pieces are unnotched and their thickness is
smaller than the standard specimens. The same material shall be tested in parallel
in a reference machine.

According to the ASTM E 23-88 standard, Charpy impact machines shall be

verified by testing with standardized specimens. If these produce values outside
the tolerances, those parts subjected to wear shall be inspected and if necessary
repaired or replaced. ASTM E 127l-88 gives very detailed instructions for manu-
 -13-

facturing and qualifying Charpy verification specimens. Certified reference

material is delivered by the National Institute of Standards and Technology. It
must be noted that this material can be used only in machines made according to
ASTM E 23. Certain dimensions in ISO and EN standards differ from those in
ASTM.

3 CHOOSING AND MAINTAINING REFERENCE MATERIALS

The ISO-GUIDE 35 “Certification of reference materials - General and statistical

principles” describes the requirements for establishing certified reference
materials. Much importance is given to confirming homogeneity of the materials
and to the statistical methods to define the uncertainties.

Certification can be established by a definite method, by interlaboratory testing or

through a metrological approach.

The ISO-GUIDE 31 “Contents of certificates of reference materials” describes the

requirements for the certificates. There are unfortunately a number of commercial-
ly available reference materials whose certificates do not clearly state the level of
confidence, level of certification or the uncertainties. The users of CRMs have to
seek confirmation on the quality of reference materials before purchase.

3.1 Choosing a reference material

The user of a reference material must decide what properties are relevant to the
measurement process. The level (of property value) of the CRM should corre-
spond with the level of the intended measurement. The matrix should be close to
the matrix of the material to be subjected to the measurement process (carbon in
low-alloy steel or carbon in stainless steel). The form should correspond with that
of the intended samples (solid, liquid, gas...). The quantity of the CRM should be
 -14-

sufficient; the need to obtain additional CRM during the experimental programme
should be avoided and batches of CRMs should partly overlap in use for continu-
ity. Stability of the CRMs should be ensured or changing of the properties should
be recorded in the certificate. The uncertainty of the certified value should be
acceptable when compared to requirements (legal limits, trade agreements and
commercial requirements, previous experience in the laboratory). Appendix 4
shows an example of a BCR reference material.

3.2 Working standards

Often the availability of certified reference material is limited, the use of the
reference material is extensive or the price of the certified reference material is
high. In these cases the commercial certified reference materials cannot be used
in everyday practice by the laboratories. The laboratories can and are also encour-
aged to produce in-house working standards.

Homogeneity of these working standard reference materials should be verified

with sufficient accuracy through in-house test series; the level of the parameters
of interest have to be “calibrated” against valid certified reference materials.
Uncertainty (95% confidence limits, for example) should be estimated for the
working standard.

Generally a well-prepared and evaluated working standard with traceability to

CRMs is of higher value to the quality assurance of a testing laboratory than a
commercial reference material with limited confidence and questionable certifi-
cate.
 -15-

3.3 Household practices

Reference materials have to be treated as highly vulnerable “instruments”. The

CRMs have to be stored in a manner which will insure that none of the properties
of the CRMs alter more than the “normal” deterioration stated in the certificates,
if evident. Records on all uses of CRMs should be kept and all unqualified uses
of CRMs have to be discouraged.

Clear procedures should be maintained on how each reference material is to be

used and what records kept. CRMs, RMs or working standards should be used in
a continual manner to ensure quality and traceability of testing in a laboratory as
described in the following chapter.

4 REFERENCE MATERIALS IN QUALITY ASSURANCE OF TESTING

LABORATORIES

Two major uses of reference materials in a testing laboratory can be identified.

One use is the indirect verification and calibration of test instruments, while the
other is the continual monitoring of laboratory performance.3

The NIST catalogue gives the following reasons for use of reference
materials:
* To help develop accurate methods of analysis (reference met-
hods);
* To calibrate measurement systems used to:
(a) facilitate exchange of goods,
(b) institute quality control,
(c) determine performance characteristics, or
(d) measure a property at the state-of-the-art limit; and
* To assure the long-term adequacy and integrity of measure-
ment quality assurance programmes.
 -16-

4.1 Indirect verification of test instruments

As mentioned above, verification of an instrument is understood as confirmation

by examination and provision of evidence that the instrument meets the specified
requirements set for it. Verification provides a means for checking that the
deviations between values indicated by a measuring instrument and the corre-
sponding known values of a measured quantity are consistently smaller than the
maximum allowable error defined in a standard, regulation or specification
characteristic of the measuring equipment.

Indirect verification usually consists of an overall checking of the working

conditions of the test instrument as well as carrying out the normal test pro-
cedures using samples of known level(s) of measured parameter(s). The results
from the tests are compared with the values given for the verification samples and
conclusions are drawn based on the deviations (differences of test results and
corresponding “true value”) and the requirements given for the instrument.

The result of verification leads to a decision either to restore to service, or to

perform adjustments, or to repair, or to downgrade, or to declare obsolete. In all,
cases it is required that a written trace of the verification performed shall be kept
on the measuring instrument’s individual record.

Often the standards that define indirect verification of test instruments require the
use of certified reference materials (CRMs) as samples with known levels of the
parameters of interest.

A reduced level of reliability can be allowed for the reference material; this
depends on the requirements set for the instruments, on the level of uncertainty of
the certified reference material (traceable, verified, calibrated material) and on the
performance of the instrument. The laboratory can manufacture its own working
standard reference materials or use reference materials without a certificate as
long as the laboratory has verified and documented the production and the
 -17-

traceability of the working standard. ISO-GUIDE 33 encourages laboratories to

study possibilities to supplement CRMs with working standards when the required
CRMs have a limited availability and relatively high cost.

4.1.1 Calibration

Calibration is often (but not necessarily) an integral part of verification of a test

instrument. Calibration defines the relationship between values indicated by a
measuring instrument or measuring system, or values represented by a material
measure, and the corresponding known values of a measurand. The known value
here can be given through CRM or RM as above and through instruments whose
calibration is traceable to national and international standards.

4.2 Controlling laboratory performance

Controlling the laboratory performance should be understood as a philosophy, a

system and a set of techniques for controlling and improving testing processes. In
a sense the process is related to the statistical process control (SPC) (Juran 1974)
generally used in the production and service industries. In SPC the normal
production is evaluated in an organized manner by measuring critical attributes of
the production afterwards.

Much of the testing is destructive or testing affects the attributes measured; often
adjustment, verification or calibration of the instruments with reference material
(or instruments) is required prior to testing. The possibility to re-test the samples
is thus very unlikely and valuable information is received on performance of the
test process by recording the measurements (of known values of attributes) done
prior to the test.
 -18-

The process described above is very open and has to be included in the normal
testing procedures. Another way to receive material for performance control is to
include blind samples in the testing process. The open process has the possibility
of biased or specially “good” practice in testing as the CRMs are naturally
identified; blind testing can be difficult to organize and in time samples of known
values of attributes are identified by the personnel. Often it is advisable to
incorporate not only testing but also sample preparation in the blind testing
programmes.

An integral part of successful control is good planning of the sampling and testing
plan along with the number of tests, levels and intervals. Control charts measure
the process over time. There are many ways to draw up these charts; below two
choices are described.

4.2.1 X - R charts

The conventional SPC control chart is constructed as a pair of graphs showing

both mean and range of measurements. Sample size for SPC charts is usually
between 2 and 10. The means and “control limits” (three standard deviations) are
calculated (Fig 1). The next step is to determine whether the process is under
control or not. A simple test sets conditions for out-of control process as follows:
* any points above the upper control limit or below the lower control
limit in either graph or
* eight points in a row above (or below) the centreline in either graph.

This test tells only if the process is under control, not if the measurements are
within required tolerances. The tolerance limits have to be added to the graphs to
justify evaluation on whether the results are within specifications.
 -19-

X-bar

8 LCL
6
4
2
0

R
2.5 | UCL

Fig. 1 X-R chart of a process showing both upper and lower control
limits set at three standard deviations from the average value
lines. Sample lot size is 5 per data point.

4.2.2 Value-time charts

A simpler and probably more used method within testing is recording individual
measurements of the attributes of known value i.e. testing CRMs. Fig. 2 shows a
hypothetical chart of this kind. A test is done each morning using a CRM of
known level and uncertainty according to procedures and standards. The graph
shows the uncertainty of the CRM and the (in-house) specification limits for the
tests.
 -20-

4.3 Using the control charts

Fig. 2 a) shows a value-time chart with measurements at equal intervals and

sample size 1, while limits of allowed deviations are also included. Fig. 2 b)
shows increasing scatter of results; this phenomenon can be due to loss of calibra-
tion/adjustment or even wear of the instruments. All in all, the verification
interval of the equipment is clearly overdue.

VALUE VALUE
12 12
10 10
8 8
6 6
4 4
2 2
0 0
0 20 40 60 80 100 0 20 40 60 80 100
VALUE VALUE .
12 12
10 10
8 8
6 6
4 4
2 2
0 0
0 20 40 60 80 100 0 20 40 60 80 100

Fig. 2 Value-time charts showing control measurement data.

Explanations of the figures in the text

Figure 2 c) shows a sudden increase in scatter in the results, which could be due
to a mechanical fault in the instrument. Further analysis of the data in figure 2 d)
shows that when the measurements by two operators are separated, the increase
in scatter is probably due to the test being conducted by an operator less skilful
than the initial operator.

A process has to be under control before the quality of the results can be
managed. Timely recording of testing performance and the process charts provide
 -21-

simple practical means of finding the level of control of the operations, the
quality of testing and pinpointing the forthcoming problems possibly causing
quality nonconformance.

4.4 Interlaboratory tests in estimating laboratory performance and the uncertainty

of testing

Interlaboratory tests can be used to study the reliability of test methods (ISO
5725), performance of testing laboratories (proficiency testing) and to estimate the
uncertainty of laboratory performance. Interlaboratory tests need to have a
relatively high number of participants to yield statistically valid results. Thus such
test programmes can be arranged only at relatively long intervals. Proficiency
testing and estimation of uncertainty of test performance can be carried out
between the interlaboratory test programmes through the use of materials whose
performance (level of parameters, repeatability, reproducibility, long-term stability
etc) has been established in statistically valid comparisons (interlaboratory tests).

Such materials can, as mentioned above, be certified reference materials. Thus

CRMs can be used in laboratories to estimate the development of uncertainty of
laboratory performance.

The repeatability value (r) defines the value below which the absolute difference
between two single test results obtained under identical (repeatability) conditions
may be expected to lie with a probability of 95%, whereas reproducibility value
(R) gives the equivalent limit value when using the same method and test material
but in different conditions (time of test, equipment, personnel, laboratory)4.

The within-laboratory long-term reproducibility value can be used as a conserva-

tive estimate for uncertainty of laboratory performance when using the test
method.

4
definition, ISO 5725
 -22-

5 SOME SOURCES OF CERTIFIED MATERIALS

There is several sources to get certified reference materials. Some of the best
known are the Community Bureau of References, BCR, the National Institute of
Standards and Technology, NIST, Laboratory of the Government Chemist, LGC
and Analytical Quality Control Services, AQCS, which is related with Interna-
tional Atomic Energy Agency, IAEA.

5.1 BCR

The role of the Community Bureau of References (BCR) programme of the

European Community is to solve the major technical difficulties which arise in the
measurements done in the implementation of Community Directives or in apply-
ing standardized methods of testing.

BCR operates through collaborative programmes (often interlaboratory test

programmes). The programmes have multiple stages with the aim of harmonizing
the performance of the participating laboratories. Large numbers of laboratories
carry out a particular test or measurement, and the inherent results with a high
level of confidence are generally used to produce reference materials (CRMs). By
autumn 1991 the BCR programme had produced in approximately 550 projects
over 300 CRMs, and the CRMs are used in over 2500 laboratories.

For the mechanical tests of metals BCR can supply a reference material for
Charpy impact testing and is planning to certify a CRM for tensile testing
(NIMONIC 75, CRM 425, which is currently available as a creep testing refer-
ence material).

If a laboratory has a specific problem due to lack of reference materials it should

contact BCR and initiate a collaborative programme on the topic.
 -23-

5.2 NIST

The National Institute of Standards and Technology (NIST) offers for sale
primarily NIST Standard Reference Materials (SRM’s). These are well-character-
ized materials produced in quantity to improve measurement science. SRM’s are
certified for specific chemical or physical properties. In their certificates NIST
reports the results of the characterization and indicate the intended use of
material.

NIST has the function to develop, produce, and distribute SRM’s that provide a
basis for comparison of measurements on materials, and that aid in the control of
production processes. To perform this function, the Office of Standard Reference
Materials evaluates the requirement of science, industry, and government for
carefully characterized reference materials, and their production and distribution.

6 COMAR DATABASE

The Reference Material Committee (REMCO) of ISO adapted and started jointly
to develop the French reference material database originated by Service de
Matériaux de Réference at the Laboratoire National d’Essais (LNE). The original
goal was to provide answers to the question “Where can I buy a CRM with the
following properties...?” without the need to consult all the catalogue s of the
many suppliers. (Klich 1991)

Code of Reference Materials COMAR has been developed by three contractors;

LNE (France) as the main contractor and LGC (UK, replaced NPL 1988) and
BAM (Germany) as associate contractors. COMAR consists of over 3500 CRMs
from 50 producers; the database contains
* name and general description of the material,
* name and address of the producer,
* form of the material and
* certified properties.
 -24-

The COMAR database is delivered on floppy disks and a standard IBM-compat-

ible (286 processor or better, min 30Mb hard disk) computer can run the database.
The user interface is rather easy to learn through a step-by-step menu system
though there seem to be still a number of intrinsic bugs and anomalities.

The database is quite comprehensive and thus searching for relevant reference
materials can be tedious. Good knowledge and experience of the database and
also the tests performed and CRMs needed is necessary to efficiently use the
database.

The database is now reaching the level of international commercial utilization.

Currently the contractors are unfortunately not able to quote uniform prices and
terms for delivery and updating the database. Hopefully an agreement will be
reached in the REMCO meeting in May 1992.

7 NORDIC QUESTIONNAIRE ON REFERENCE MATERIALS

In order to get information on the general knowledge of reference materials and

on use of the RMs in testing laboratories (destructive testing of metals) a ques-
tionnaire was sent to 82 testing laboratories in the Nordic countries. Questions on
the use of testing standards and reference materials for 12 different test methods
was asked. Answers were received from 16 laboratories.

The distribution of the test methods and the reference materials used in these
laboratories is presented table 1.
 -25-

Table 1 The test methods used in the laboratories

Test Number Number of labs

of laboratories using RMs

Brine1l hardness testing 14 9

Rockwell hardness testing 13 9
Vickers hardness testing 16 9
Tensile testing 13 2
Compression testing 5 -
Bend testing 12 -
Flattening testing 4 -
Charpy impact testing 13 4
Drop weight testing (DWT) 4 -
Drop weight tear testing (DWTT) 2 -
Fatigue testing 7 -
Fracture mechanical testing 7 -

According to the answers, reference materials are quite widely used in hardness
testing and most of reference materials had some certification; there were 7 supp-
liers. Most of the certificates did not fully correspond with the ISO GUIDE 31-
1981 “Contents of certificates of reference materials”. Some certificates referred
to national standards (DIN, BSI) and national requirements and were thus mainly
intended for local use; certificates were not very clear in general.

The most popular (and generally also the most trusted) hardness reference blocks
come from Staatliches Materialprüfungsamt Nordrhein-Westfalen (MPA-NRW),
whose certificates give good information on the reference blocks and their use.
Calibration (and certification) of these blocks is according to DIN 51303.

The British and the French systems have accredited (or recognized) suppliers of
reference blocks and some of the certificates (most of the new certificates) refer
to the accreditation.

Many laboratories used reference blocks with no certificates; some RMs were
rather old, dating from the 1970’s, so the use of the blocks had obviously not
been very frequent.
 -26-

One laboratory manufactures in-house reference materials (working standards) for

tensile and Charpy impact testing (RMs according to ISO 442) and supplies one
participating laboratory with tensile test reference material.

There were a number of accredited laboratories from different Nordic countries

among the respondents. Some of these laboratories did not use reference materials
or used uncertified RMs; one laboratory stated that their equipment is traceably
verified once yearly according to standard verification methods and thus no
reference material is needed for quality assurance of testing. These answers
definitely show that the accreditation requirements between the Nordic countries
vary significantly as in some countries the use of CRMs in indirect verification
and between-verification control of equipment is compulsory.

The questionnaire showed that the significance of the use of reference materials
is not very well understood or even identified as a part of quality assurance
activity in a testing laboratory. Availability of reference materials and of general
information on reference materials seems very limited.

8 CONCLUSIONS

The reference materials are an integral part of the quality assurance of testing
laboratories. Test performance between laboratories can be harmonized through
use of certified reference materials, and individual laboratories can continually
follow up their own performance and check the instruments between verifications
and calibrations. Reference materials are used also to calibrate and tune instru-
ments which are difficult or impossible to calibrate with direct methods. RMs are
used as a transfer standard between a high-quality instrument and a working
instrument.
 -27-

Recommendation: The importance of attention to quality and traceability of

measurements of testing laboratories cannot be stressed enough. Use of
reference materials in periodic checks, verifications and calibrations is an
essential part of quality assurance. Laboratories should judge for them-
selves how to organize the process; accreditation bodies should stress the
significance of RMs and if possible publish supportive information.

Related standards

There is a definite incompatibility between quality standards (EN 45001, ISO

GUIDE 25), testing standards and the quality of reference materials and partly
also the application of EN 45001 by the national accreditation bodies.

Recommendation: Test methods should be harmonized with the quality standards

and also the available reference materials (requirements in standards have
to be related to physical possibilities and available uncertainty).

Availabilitv of reference materials

The number and quality of reference materials are increasing, although informa-
tion on them has not been spread so well. The active role of BCR has had a
positive effect on the field; bringing laboratories and RM producers (Appendix 5)
together in collaborative programmes will increase knowledge and interest in the
related questions. All CRMs should be produced in a way which would allow for
international use and distribution, preferably through BCR in Europe.

Recommendation: International distribution of CRMs is encouraged; COMAR,

for example, should be updated and used actively.
 -28-

Certification documents

Some of the reference materials available are still inadequately documented -

uncertainties are not given and it is difficult to know whether the material has
been adequately studied and the RM produced in a suitable manner. This uncer-
tainty has a negative effect on the willingness of laboratories to use reference
materials.

In some countries RMs are produced by accredited or national calibration labora-

tories according to national standards. Information on the certification/calibration
methods is generally available nationally but not internationally.

Recommendation: The producers of RMs and CRMs should be encouraged to

publish certificates according to ISO GUIDE 31 to help users (laborato-
ries) evaluate and compare possible CRMs.

COMAR database

The COMAR database is an effective and efficient way to find relevant certified
reference materials. The database should be at least nationally available (in France
30 Minitel subscribers are currently connected to COMAR). There are two major
concepts of organizing the use of COMAR nationally; on one hand a national
body (accreditation body, national technical information centre etc) could centrally
be responsible for giving out information on reference materials, on the other
hand national laboratories would know better their own technical field and could
thus better serve testing laboratories. A national solution is, however, necessary
to spread the information to all laboratories needing information on reference
materials.
 -29-

REFERENCES

ASTM E 23-88. Standard test methods for notched bar impact testing for metallic
materials.

ASTM E 127l-88. Standard practice for qualifying Charpy verification specimens

of heat-treated steel

EN 45001:1989, General criteria for the operation of testing laboratories

ISO 442-1965. Verification of pendulum impact testing machines for testing

steels.

ISO 9001:1987, Quality systems. Model for quality assurance in

design/development, production, installation and servicing

ISO/IEC GUIDE 25:1990 (E) General requirements for the competence of

calibration and testing laboratories

ISO GUIDE 30:1981 (E), Terms and definitions used in connection with reference
materials

ISO GUIDE 31:1981 (E), Contents of certificates of reference materials

ISO GUIDE 33:1989 (E), Uses of certified reference materials

ISO GUIDE 35:1989 (E), Certification of reference materials - General and

statistical principles

Juran, J.M. 1974. Quality Control Handbook. 3rd ed. New York, McGraw-Hill
Book Company. 1780 p.
 -30-

Klich, H. 1991. COMAR Database for Certified Reference Materials. Berlin,

BAM. 4p.

Klich, H. Personal communication and COMAR demonstration, 31.1.1992. Berlin,

BAM.

NAMAS Accreditation standard M10, General Criteria of Competence for

Calibration and Testing Laboratories. 1989. 24p.

NAMAS Information Sheet NIS 13, Traceability, hardness measurements. 1987.

5p.

NIST Standard Reference Materials Catalogue 1990-91, NIST Special Publication

260, U.S. DEPARTMENT OF COMMERCE, National Institute of
Standards and Technology

prEN 10 045 Part 2:1990, Metallic materials - Charpy impact test - Part 2:
Verification of the testing machine (pendulum impact)

Quality Management in the Laboratory, Espoo, Finland, 24. - 25.9.1991, National

Association of Testing Authorities, NATA Australia in conjunction with
the Technical Research Centre of Finland, VTT.
 Excerpts from BCR REFERENCE MATERIALS CATALOG
Appendix 1 1(2)

INTRODUCTION

For some ten years the Community Bureau of Reference (BCR), a department of the Commission of the
European Communities, has provided laboratories with a means of carrying out collaborative measurement
studies in Physical Measurements and Chemical Analysis.

To understand the role of the BCR it is important to remember that common written standards (Norms),
although essential to achieve harmonization of measurements, are not sufficient by themselves to avoid
discrepancies of results.

The principle of this contribution is to bring together some of the most expert laboratories of preferably all the
Member States for intercomparison exercises. Results are discussed with all participants. Discrepancies are
given particular attention to identify their causes. Subsequently, additional measurements are made to verify
that the causes of errors can be eliminated. When necessary several successive intercomparisons are done to
study the effect of the different parameters.

Certification - Uncertainties - Uses

When the major sources of errors are eliminated and agreement is reached on a value which can be considered
as the most accurate in the current state of the art, it is essential to provide a means of maintaining the
consistency between laboratories and of disseminating the accurate value to other laboratories. The instruments
or pieces of equipment used to transfer values are called transfer standards in Applied Metrology and reference
materials in Analytical Chemistry.

The Community Bureau of Reference undertakes the establishment of transfer standards and reference materials
which can be used by any laboratory which wishes to verify its methods and calibrate its instruments.

The uncertainty is by definition the estimate of the range of values within which the true value is most likely
to be. The uncertainties given in this catalogue take into consideration the variations between laboratories and
between methods and care is taken not to underestimate uncertainties. They are given at a confidence level of
95%.

A certified reference material can be used by a laboratory to verify if the method used gives results in agreement
with the certified values. If the result is different the user is immediately warned that he has to look for errors
in his method and to find their causes.

For calibration the reference material is used to assign a value to the indication of his instrument.

Reference materials are used mainly for:

- calibration and verification of measurement processes in routine analytical conditions;
- internal quality control and quality assurance schemes;
- verification of the correct application of standardized methods;
- development and validation of new methods of measurements.

The BCR programme welcomes any suggestion from industrial organisations, official laboratories, research
establishments or universities, concerning measurements of practical importance that need to be improved or
for which there is a need for reference materials. Further information on the BCR is available on request.

Report

Each sample is usually delivered with a certificate and a report which contains all the data which formed the
basis for the certification. It also gives details on the methods of measurements that were used, recommen-
dations on the best methods to use, and advice for the use of the material.
 Excerpts from BCR REFERENCE MATERIALS CATALOG
Appendix 1 2(2)

Procedure for purchase of BCR reference materials

The certified reference materials which are made available by the BCR were established in response to a need
to help laboratories to improve their measurements. Except when batches of reference materials are very limited,
the BCR makes no restriction in their availability for these specific uses as described in its catalogue.

The BCR cannot supply quantities of these materials for any other types of uses.

1. Purchase orders for reference materials and requests for information should be addressed to:

COMMUNITY BUREAU OF REFERENCE (BCR)

COMMISSION OF THE EUROPEAN COMMUNITIES
200, Rue de la loi
B-1049 BRUSSELS

or by telexing:
BCR -BRUSSELS
21877 COMEU B

All purchase orders should give: CRM numbers indicated in the catalogue, name of the material and number
of units required.

2. Delivery time: 3-4 weeks maximum after receipt of the purchase order by the BCR.

3. Shipment: prices include handling and normal postage (surface mail, air mail). If airfreight is requested, it
has to be paid by the customer.

4. Terms of payment: the payments shall be made preferably by bank transfer following instructions to be
found in the debit note (or invoice) that the customer will receive after delivery of the samples.
Methods of payments such as credit letter are not acceptable. Prices of reference materials are shown in
the price list attached. Payments are expected within 30 days of receipt of the debit note.
 Appendix 2 l(2)

COMMISSION OF THE EUROPEAN COMMUNITIES

COMMUNITY BUREAU OF REFERENCE - BCR N°s. 27 I to 56E

CERTIFIED REFERENCE MATERIAL

CERTIFICATE OF MEASUREMENT

Batch E

CRM 016

IMPACT TOUGHNESS
OF V-NOTCH CHARPY SPECIMENS
(Energy level 120 J Nominal)

Impact energy at 20° C Cenified value (1) Uncertainty (2)

According ASTM E 23-82 124.8 J ± 5.0J

According ISO 121.5 J ± 4.7J

(1) This is the mean impact energy for the 5 specimens

delivered in one set

(2) The uncertainty includes mainly the effect of the

variation between samples

DESCRIPTION OF THE REFERENCE MATERIAL

A unit comprises 5 specimens which must be broken successively in the machine to verify.
The specimens are packed in protective oil and cleaning must done as recommended.
 Appendix 2 2(2)

INSTRUCTTONS FOR USE

Specimens should be kept in their original packing until they have to be used.

Special attention is drawn to cleaning, for which vigorous methods should be avoided
(e.g. brushes or ultrasonic). The following is recommended :

1. Wipe excess oil with cellulose paper

2. Immerse specimens in a clean bath of degreasing solvent for about 5 minutes
3. Wipe the specimens with cellulose paper
4. Immerse the specimens in a bath of fresh absolute alcohol
5. Wipe with paper and allow to dry in still air
6. Keep at room temperature for 24 hrs prior to testing

PARTICIPATING LABORATORIES

National Physical Laboratory, Teddington (UK)

Material Testing Laboratory, Watertown (USA)
Cogne, Aosta (IT)
Franco Tosi Industriale, Legnano (IT)
Cermet, Bologna (IT)
Equipos Nucleares SA, Santander (ES)
Bundesanstalt für Materialforschung und-prüfung, Berlin (DE)
Staatliches Materialprüfungsamt, Dormund (DE)
Ugine, Isbergues Research Centre (FR)
Laboratoire National d’Essais, Paris (FR)
Aubert et Duval, Gennevilliers (FR)
Aubert et Duval, Les Ancizes (FR)
Centre de Recherches et d’Etudes d’Arcueil (FR)
Sollac, Dunkerque (FR)
GTS Industries, Dunkerque (FR)

METHODS USED

Charpy Testing Machine according to ASTM E23-82

Charpy Testing Machine according to ISO (R 442)

LEGAL NOTICE

This document was prepared under the sponsorship of the Commission of the European
Communities. Neither the Commission of the European Communities nor any person
acting on its behalf makes any warranty or representation, express or implied, that the
use of any information, material, apparatus, method or process disclosed in this document
does not infringe privately owned rights ; or assumes any liability with respect to the use
of, or for damages resulting from the use of, any information, material, apparatus,
methods or process disclosed in this document.

CERTIFICATION REPORT

The work done to certify this reference material is described in a report which is
available to users free of charge upon request.

Provisional address : rue de la Loi 200, B-1049 Brussels - Phone (32 2) 235.50.14 - Fax (32 2) 235.80.72
 Appendix 3 l(1)

Producers of RMs and CRMs:

- Acco Industries Inc. (ACCO)
- The American Oil Chemists’ Society (AOCS)
- Analytical Quality Control Services (AQCS)
- Centre Mécanique Chimie Matériaux (CMCM)
- Community Bureau of Reference (BCR)
- Eidgenössische Materialprüfungs- und Forschungsanstalt (EMPA)
- Engineering & Scientific Equipment Ltd. (ESE)
- Forschungs- und Materialprüfungsanstalt Baden-Württemberg (FMPA)
- Foundrax Engineering Products Ltd
- Georg Reichter, Eslinger/Neckar
- Laboratoire National d’Esaais (LNE)
- Laboratory of the Government Chemist (LGC)
- National Institute for Environmental Studies (NIES)
- National Institute of Standard and Technology (NIST)
- National Research Centre for Certified Reference Materials (NRCCRM)
- National Research Council Canada (NRC)
- Shimadzu Kyoto
- Staatliches Materialprüfungsamt Nordhein-Westfalen (MPA NRW)
- Werkö
- Yamato Scientific
TECHNICAL REPORTS FROM THE PROGRAMME GROUP FOR QUALITY ASSURANCE

082 Larsen, B., NORDTEST Guidelines for development of Quality Manuals for testing laboratories. Helsinki
1987. Nordtest, NT Techn Report 082. 60 p. (in English and Danish)

084 Fahlén, P., Madsen, E., Johansen, M.M., Rautiainen, L., Siitonen, V., Herje, J. & Sandberg, P.I.,
Guidelines for development of Quality Manuals in the field of building physics, heating & ventilation
and adjacent technical areas. Helsinki 1987. Nordtest, NT Techn Report 084. 25 p.

104 Häyrynen, J., Draft NORDTEST guidelines for interlaboratory test comparison, Espoo 1988. Nordtest,
NT Techn Report 104. 95 p.

105 Fremstad, D., Lundgren, B., Olsen. H. & Saarela, K., Guidelines for development of Quality Manuals in
the field of chemical analysis. Espoo 1988. Nordtest, NT Techn Report 105. 32 p.

107 Haglund, M., A check list concerning the cost/benefit of laboratory accreditation. Espoo 1989.
Nordtest, NT Techn Report 107. 18 p. (in Swedish)

107:E Haglund, M., Check-list concerning the advantages, cost and disadvantages of laboratory
accreditation. Espoo 1990, NT Techn Report 107 Engl, Edition 2. 9 p.

118 Häyrynen, J., Interlaboratory testing. Espoo 1989. Nordtest, NT Techn Report 118. 11 p.

119 Quality assurance in testing laboratories, Espoo 1989. Nordtest, NT Techn Report 119. 156 p. (in
Dan/Nor/Swed/Engl)

131 Quality assurance in testing activities. Espoo 1990. Nordtest, NT Techn Report 131. 261 p. (in
Dan/Nor/Swed/Engl)

143 Larsen, J.B., An examination of the costs in establishment and maintenance of an accreditation and
the including quality control system. Espoo 1991. Nordtest, NT Techn Report 143. 106 p.

144 Ploug, H.H., Nordtest practical guidelines for round robin tests. Espoo 1991. Nordtest, NT Techn
Report 144. 28 p.

145 Lindskov Hansen, S., Guidelines for the development of software to be used in test and measuring
laboratories. Espoo 1991. Nordtest, NT Techn Report 145. 46 p.

146 Ohlon, R., About procedures for internal quality audits of testing laboratories. Espoo 1991. Nordtest,
NT Techn Report 146. 39 p.

147 Andersson, H., An introductory study of calibration in testing laboratories. Espoo 1991. Nordtest, NT
Techn Report 147. 30 p.

148 Änkö, S. & Sillanpää, J., Requirements on the personnel and organization of testing laboratories.
Espoo 1991. Nordtest, NT Techn Report 148. 70 p. (in Swedish)

149 Forstén, J., A view on the assessment of the technical competence of testing laboratories. Espoo
1991. Nordtest, NT Techn Report 149. 46 p.

164 Sillanpää, J., Requirements of ISO 9002 for testing and inspection. Espoo 1992. Nordtest, NT Techn
Report 164. 42 p.

165 Baade, S., Interpretation of EN 45 001 for fire testing laboratories, Espoo 1992. Nordtest, NT Techn
Report 165. 25 p.

177 Häyrynen, J., Kallio, H. & Kosonen, A.-M., Reference materials in quality assurance of testing
laboratories. Espoo 1992. Nordtest, NT Techn Report 177. 35 p.

178 Änkö, S. Qualification system for the personnel of testing laboratories. Espoo 1992. Nordtest, NT
Techn Report 178. 17 p.

179 Ohlon, R., Comparison of standards with requirements on calibration and testing laboratories, Espoo
1992. Nordtest, NT Techn Report 179. 71 p.
nordtest
Mailing Address: Tekniikantie 12 FIN-02150 Espoo, Finland. Phone +358 9 455 4600, fax +358 9 455 4272
www.nordtest.org