The Standard
Vol. 15, 1 The Journal of the Measurement Quality Division, American Society for Quality Spring 2001/Winter 2000
IN THIS ISSUE
Editor’s Column ............... 1
Past Chair’s Column ........ 4
Website Update ................ 6
Contrarian Metrologist ..... 6
The Learning Curve ......... 8
CCT Update .................... 10
Inspection Division ........ 12
12 Steps to Useful Software
Metrics ............................ 13
Trusting Measurement
Results in the Chemical &
Process Industries ......... 22
Profile on: NCSL
International ................... 23
Pythagorean Theorem to
the Rescue ...................... 26
Upcoming Events ........... 34
A2LA News Update ........ 35
World of Metrology ........ 37
2001 Measurement Quality
Conference ..................... 38
Met/Tech Books ............. 39
Measurement Uncertainty
Ref. & Guidance Doc. .... 43
Measurement Uncertainty
Training & Consulting
Sources ........................... 44
Membership Report ....... 45
MQD Officers .................. 46
Regional Councilors ...... 47
Congratulations NIST
on your 100th Anniversary
The Editor’s Column
Straight Talk on
Measurement:
by Frank Voehl
“Tout est complex dans la nature”
It is a pleasure to be contributing to The
Standard as The Editor once again. Our Divi-
sion is the measurement and metrology arm of
the ASQ organization devoted to the develop-
ment and use of a metrology-related quality
systems perspective, measurement system
dynamics methodology, and “learning metrics”
advancement throughout North America and
around the world. It is an exciting time to be a
member of the ASQ and the QMD.
There are many international sections
(Spain, Japan, China, Japan, Italy and India to
name a few) that are spawning measurement
oversight bodies, and more are in the process
of being created.
With our practitioner viewpoint and profes-
sional administration, the Society has now
more to offer its members. In addition to the
Quality Progress magazine, technical journals,
and various Division newsletters, the annual
international conferences and the President’s
Newsletter, members can obtain information
such as the Bibliography, historical data on the
Society, and high school and college teaching
materials on the Internet. There are also sev-
eral electronic discussion lists for different in-
terest groups.
A sub-committee of the GTC is looking
further into the issue of how we can make fuller
use of electronic media—including electronic
meetings, management of our web site and
web-based conference proceedings. The So-
ciety finances, based on annual dues and
conference income, look stable. Additional in-
come is being produced by the Society office’s
sale of its many books and previous
publications. Another sub-committee is inves-
tigating how to make best use of the budget
surplus, such as offering best student paper
awards, conference travel support, etc.
So, things are in good shape in general.
What about the “challenges?” I will now spend
some time on a few conceptual and technical
challenges that, in my view, we measurement
quality dynamicists must deal with in the near
future.
1. The first challenge has to do with choos-
ing a good measurement problem to work on.
Many years ago, when I was an under-
graduate student, I recall my professors advis-
ing: “A good industrial engineer (metrologist) is
the one that examines a given problem and
chooses the right modeling technique/tool from
his/her toolbox; do not walk around with a
screwdriver, looking for a screw that would fit
your tool! Adjust yourself to the problem.”
Over the years, I gradually reached the
conclusion that this advice was wrong at a very
fundamental level. I have gradually learned
that good modeling primarily and precisely
starts with looking around and finding a “good”
problem that would fit our screwdriver. I submit
that much care must be exercised in “choosing
the metrology problem,” more specifically in
identifying, interpreting, shaping and even “de-
fining” it. If a good measurement feedback
Continued on page 3
 Page 2 The STANDARD Spring 2001/Winter 2000

The STANDARD
The Journal of the Measurement Quality Division
Advertising
www.metrology.org
Submit your draft copy to Frank Voehl, the
Publication Information
Advertising Manager, with a request for a The STANDARD is published quar-
quotation. Indicate size desired. Specify terly by the Measurement Quality Divi-
whether you will provide camera-ready sion of ASQ; deadlines are March 15,
copy or desire that we produce final copy. June 15, September 15 and January
Publication 15.
The following rates are for the space only.
Staff Copy preparation and typesetting will be Input for text material by email or on 3
1/2" diskette in Microsoft Word saved
Executive Editor extra, if provided by the The STANDARD.
in Rich Text Format (RTF). If it is not
Frank Voehl Business card size ................... $100 feasible to send text in electronic form,
St. Lucie Press 1/8 page ................................... $150 clean printed text can be submitted.
280 Lake Drive
1/4 page ................................... $200 Graphics or illustration material can be
Coconut Creek, FL 33066
Tel: 954-972-3012 1/3 page ................................... $250 sent in eps, tif, pict or jpeg format.
Fax: 954-978-0643 1/2 page ................................... $300
Photographs of MQD activities or
E-mail: FVoehl@aol.com Full page .................................. $550 people would be especially appreci-
Associate Editor Advertisements will be accepted on a “per ated.
Europe and Asia issue” basis only; no long term contracts Publication of articles, product re-
John Shade will be available at present. leases, advertisements or technical
Good Decision Ltd. Advertising must be clearly distinguished information does not imply endorse-
Dunfermline, KY11 3BZ as an ad. Ads must be related to measure- ment by The STANDARD or the Mea-
Scotland ment quality, quality of measurement, or a surement Quality Division of ASQ.
Tel: +44 1383-733553 related quality field. Ads must not imply While The STANDARD makes every
Fax: +44 1383-733588 endorsement by the Measurement Qual- effort to ensure the accuracy of ar-
E-mail: js@gdl.co.uk ity Division or ASQ. ticles, the publication disclaims respon-
Associate Editor sibility for statements of fact or opinion
U.S. and Canada
Letters to the Editor made by the authors or other contribu-
Mark Schoenlein The STANDARD welcomes letters from tors.
P.O. Box 206 members and subscribers. We offer the
following guidelines. Letters should clearly Material from The STANDARD may
Perrysburg, OH 43552 not be reproduced without permission.
Tel: 419-247-7285 state whether the author is expressing
opinion or presenting facts with support- Copyrights in the United States and all
Fax: 419-247-8770 other countries are reserved.
E-mail: mark.schoenlein@ ing information. Commendation, encour-
owens-ill.com agement, constructive critique, sugges- © 2001 ASQ, MQD. All rights reserved.
tions, and alternative approaches are ac-
Advertising Manager cepted. Berating is not appropriate. If the
Frank Voehl content is more than 200 words, we may
St. Lucie Press delete portions to hold that limit. We re-
280 Lake Drive serve the right to edit letters and papers.
Coconut Creek, FL 33066
Tel: 954-972-3012 Information for Authors
Fax: 954-978-0643 The STANDARD publishes papers on the
E-mail: FVoehl@aol.com quality of measurements and the mea-
Publications Chair surement of quality at all levels ranging
J.L. Madrigal from relatively simple tutorial material to
Oxford Worldwide Group state-of-the-art.
1045 South Orem Blvd. Papers published in The STANDARD are
Orem, UT 84058 not referred in the usual sense, except to
Tel/Fax: 801-235-1899 ascertain that facts are correctly stated
E-mail: jim_oxford@msn.com and to assure that opinion and fact are
clearly distinguished one from another.
The Editor reserves the right to edit any
paper.
 Spring 2001/Winter 2000 The STANDARD
Continued from page 1
Editor’s Column
problem is chosen, then the system is much more likely
to be a success. On the other hand, an ill-chosen
problem (a static or open-loop problem) can be much
better handled by other methodologies, and a mea-
surement quality study would most likely be ineffective.
(The same general rule should naturally be applied to
any other modeling discipline.) I see this as a chal-
lenge because I have not only seen many “ill-chosen”
studies over the years but also have personally been
involved in a couple of such projects.
I thus challenge myself and all measurement quality
dynamicists to be aware of this crucially necessary
factor for a successful project and spend effort to find
good problems (similar to the “well-posed problem” of
the mathematician). The world has more than enough
problems—both in quantity and in diversity. Let us look
around for those screws and bolts and nuts that mea-
surement quality dynamics, our unique tool, would best
fit.
For example, are electromagnetic fields really dan-
gerous? Must you trust your local school system to get
rid of asbestos used for insulation decades ago? How
necessary are all the community environmental pro-
grams on which we spend over $140 billion a year?
Questions like these reflect the current and growing
concern about the complex relationships between sci-
ence and public policy, public panic and explosively
expensive costs. The conviction is spreading among
our communities that a combination of misunderstood
science, dogmatic environmentalism, sensationalizing
media, gullible public and scientifically naive legisla-
tors and regulators may be wasting billions of dollars,
causing unnecessary fears, and misdirecting the search
for the real hazards in our community lives. “Tout est
complex dans la nature.”
2. The next challenge is about the role of models in
measurement analysis and design.
In recent years, we have been witnessing compara-
tive evaluations of metrology dynamics and systems
thinking. Simultaneously, we have been debating about
a similar dichotomy between “qualitative” and “quanti-
tative” (formal-model-based) approaches. I submit that
a productive way of restating—and perhaps resolv-
ing—these dichotomies is to take a look at the relation-
ship between “measurement quality systems model”
and “system dynamicist.” The ultimate set of insights
offered by a system dynamicist about a dynamic feed-
back problem is in general much larger than the results
directly “produced” by the model. Through the exercise
of building and analyzing many measurement models,
the analyst acquires a rich set of dynamic analysis
skills. This is not to undermine the importance of
Page 3
models; it is just that the “last” model cannot by itself do
the whole job. That is probably why to most critics a
given model typically looks “unrealistic” (i.e. lacking
many intangible factors that they believe are important
in addressing the problem).
This type of criticism may tempt the modeler to
make the measurement model larger, more and more
detailed. But this path often makes the situation worse
rather than resolving it: new additions to the model will
still not satisfy the critic; and the model now becomes
large, complicated and unrealistic! The crucial factor
ignored in this debate is the fact that the ultimate
analysis and results are helped by many measurement
models that the analyst has in her repertoire. So, what
is the way out of this dilemma?
I believe the key is to make systematic use of our
mental models and dynamic insights and go through
extensive sessions of model simplification so as to end
up with a final model that is as compact as possible, yet
able to explain the fundamental dynamics. Metrology-
based systems thinking and qualitative system dynam-
ics can play an important role in this process, and it is
in this sense that they can complement measurement-
system dynamics modeling projects. Toward the
completion of a study, the analyst acquires a dynamic
understanding of the problem that she did not have in
the early phases. I therefore submit that there must be
an additional, final formal step in measurement system
dynamics modeling: model simplification. The study
cycle must be completed by the analyst crafting a much
simpler, yet fundamental, version of her “working”
model. This means substantial time and effort, but I
believe it is well worth it. The final model would be much
more likely to be put in use, published and dissemi-
nated. “Tout est complex dans la nature.”
3. This takes me to the third challenge: Low record
of metrology publications.
I have kept historic yearly data on metrology-related
publications. The general behavior consists of a boom
until the late 1980’s, a sharp decline between ‘90-’95,
a second (smaller) boom after ’95 and finally another
decline after 1998. There are of course many causes
and mechanisms behind this undesirable behavior,
beyond the scope of this newsletter. I just want to give
two brief messages: First, we should simply recognize
that there is a problem and start working on it, both as
a Division and as individuals. Second, I believe that
one important cause of the low publication record is the
fact that many of us do applied work, involving large-
scale detailed measurement models. Such models are
very difficult to publish in a scientific format. They must
first be drastically simplified and reduced in size and
detail. Many of us may be unwilling to go through this
time-consuming step. But this extra step is definitely
 Page 4 The STANDARD
worth the effort, as I mentioned above in item 2.
Model simplification is not only needed for publica-
tion but must actually be seen as an integral part of the
project itself. The publication challenge is more impor-
tant for the academians (for promotion), but let us keep
in mind that publishing, disseminating our work is also
crucial for the recognition of our field in general. “Tout
est complex dans la nature.”
4. Finally, the last challenge is improving our com-
munication links, both within ASQ and with the mea-
surement communities at large.
I believe that we face several communication chal-
lenges, within the field and between us and other fields:
We now have significant measurement quality activi-
ties in many different geographical regions—in many
countries on almost all continents. The field started at
MIT, and for many years an overwhelming portion of
measurement quality dynamics activity occurred there
and in the Northeast and Western USA. Although MIT
is still the center of activity, many other locations
around the world have recently become increasingly
more active. The trend is expected to continue; and it
is of course a positive development, provided that the
growth is healthy. In this process of growth and geo-
graphical dispersion, there are several communication
risks: there may be communication delays which can
be large relative to the speed of the growth; there may
be linguistic (in the technical sense) communication
problems, due to different communities adopting differ-
ent technical “jargon,” and there may be cultural com-
munication differences. In short, we will increasingly
face the challenge of setting up and managing effective
communication channels, the most important one be-
ing proper use of electronic communication.
Another similar intra-field communication challenge
has to do with the four constituencies of community
dynamics that our Past President Ron Kingen talked
about in his ’99 speech in Indy: to researchers, educa-
tors, consultants and practitioners. (I urge the Society
Past Chair’s Column…
Mark J. Schoenlein
I would like to take a mo-
ment to thank those indi-
viduals who responded to
our request for help. We
are fortunate to have re-
ceived offers from such a
qualified group of volun-
teers. I have included their
bios for your reading. Wel-
come aboard!
Spring 2001/Winter 2000
members to read Kingen’s talk especially if they were
not at the Indy conference.) Although these four con-
stituencies do share the general mission of the ASQ
Society, each has different specific goals, needs, prob-
lems and even jargon. Bringing these different con-
stituencies together at conferences and other meet-
ings is necessary but not sufficient. The challenge is to
create a synergetic measurement community environ-
ment so that both the metrologists and each individual
constituency benefit from it.
There is also a communication weakness between
our Society and other sister disciplines. We need to be
more actively involved at other conferences and semi-
nars, announce our activities more aggressively in
their publications, and pursue more inter-disciplinary
projects. We must also try to publish in a wider variety
of journals (not instead of but in addition to Quality
Progress and ASQ Journals/newsletters, of course).
But whatever the reason, I believe that the current
situation constitutes an unnecessary communication
handicap for the Society. In particular, it is totally
unacceptable for our models and methodology to be
reduced to competing with other “quality” organiza-
tions. “Tout est complex dans la nature.”
In closing, I believe that the QMD of ASQ is in
relatively good shape in spite of recent declining mem-
bership, as you read in the “good news” part of my
column. And I believe it would be in much better shape
if we can tackle these challenges in the near future. I
know that these are not easy tasks; they present many
difficult sub-problems. But as the French philosopher
Alain said: “tout est complex dans la nature” (all is
complex in nature). And strangely enough, we mea-
surement quality dynamicists like this complexity, for it
is this heroic-type metrology problem-centered char-
acter that makes us truly unique!
Frank Voehl, Editor
FVoehl@aol.com)
Keith Conerly, Programs
Keith has been employed with the Dow Chemical
Company since 1988. He is currently the Quality Coor-
dinator, ISO Management Representative, and Busi-
ness Analytical Leader in Liquid Separations. His re-
sponsibilities include auditing, training, and analytical
method validation, correlation, and global coordina-
tion. Prior assignments include QA and Analytical
Specialist, Senior Research Chemist, Research Chem-
ist, and Chemist in various company departments.
Keith earned a BS in Chemistry from Jackson Univer-
sity and a MBA from Central Michigan University. He
has been an ASQ member for 4 years including mem-
 Spring 2001/Winter 2000 The STANDARD Page 5

bership in the Measurement Quality, Human Develop-
ment, Chemical & Process, and Quality Audit Divi-
sions.
Joseph Filipowicz, Secretary
Joe is presently the RMA Manager for Alliance Com-
pressors, a division of Copeland and Emerson Electric,
in Natchitoches, Louisiana. His responsibilities include
SQA, receiving inspection, and gage calibration. His
29 years of experience include employment with Mather
Metals, True Temper Garden Tools, Mazda, Rockwell
International, Cummins Engine, and Tower Automo-
tive. Joe is a graduate of Gannon University in Erie,
Pa.. He has been a member of ASQ since 1981 and is
a CQT, CQA, and CQE.
Nebraska. He is the Vice Chair of ASQ Section 1302.
Tom is a lead examiner for the state of Nebraska’s
Edgerton (Baldrige based) Award and a certified ISO
9000 Lead Auditor. He has spent the last two years
focusing on return on investment projects and course
development. Before Tom moved into academics he
was in the United States Air Force for 15 years. During
that time he was involved in the development of metrics
for large computer software projects and the redesign
of budgetary systems. Prior to that Tom was an internal
Quality Consultant and Project Manager for the Air
Force Weather Agency. Tom is an ASQ certified Qual-
ity Manager and is currently pursuing a Ph.D in Man-
agement and Decision Sciences.

Samuel Windsor, Vice Chair for Regions

Kathy Hoath, Membership
Sam has 18 years of experience in Quality Manage-
Kathy is the President of Superior Technical Training ment, Engineering, Calibration, and Inspection posi-
and has worked with over 150 companies in the devel- tions. He has worked in the Defense, Pharmaceutical,
opment of the quality (business) systems. She has Telecommunications, and Commercial Equipment in-
also done extensive training in calibration, metrology, dustries. Sam holds a BS degree in Engineering and
blueprint reading, problem-solving, and continuous MS degree in Business. He holds ASQ certifications in
improvement. Kathy has owned her own company Quality Engineering, Management, Auditing, and is a
since 1990, and is very dedicated to ensuring that the Six Sigma Black Belt.
companies she works with increase their profit margins
and meet their defined goals during improvement
projects. Kathy has brought two of her own companies,
Calibration Services and Designed Outputs, to ISO
9000 registration. She has more than 20 years of
manufacturing experience and is an IRCA Certified
ISO 9000 Lead Assessor. She is also a certified lead
assessor for QS-9000. Kathy is also a member of the
US Tag to TC 176 and ISO 10012 writing group.

Nouman Ali Khan, Region 25 Councilor

Nouman is employed as the Head of Quality Control by
Descon Engineering Limited which is an ISO certified
engineering, construction, and manufacturing com-
pany operating in Pakistan as well as Abu Dhabi, and
Oman. He holds a Bachelors degree in Aerospace
Engineering from PAF College and a MBA from The
International University. Nouman has 15 years of ex-
tensive experience and training in military aircraft main-
tenance and NDT. He has spent the last 4 years
working in commercial inspection and quality posi-
tions. He is a member of ASQ, ASNT, PSNT, BINDT,
ASME, IAQP, Royal Aeronautical Society, and Insti-
tute of Engineers.

Thomas Myers, Region 13 Councilor

Tom is the Director of Corporate Assessment and a
part-time professor at Bellevue University in Bellevue,
 Page 6 The STANDARD
Website Update
The Measurement Quality Division homepage at
www.metrology.org is undergoing extensive repairs.
Thus far the volunteer contact list and division confer-
ence information have been updated. Copies of the
division’s “Standard” newsletter will be available in
PDF format. Links to other related websites are being
added as well as a few new features. The site currently
averages 15 hits per day.
Get Well Soon
to
Dan Harper & Rolf Schumacher
The Contrarian Metrologist
Making Your Assessor Happy
by Philip Stein
Everyone in the Western hemisphere seems to know
by now that I am a lead assessor for A2LA, specializing
in the application of ISO/IEC 17025 (and its predeces-
sor, Guide 25) to calibration labs. Apparently, from the
talk I’ve heard, some consider an assessment with me
to be a difficult experience. It doesn’t seem that way to
me, but of course I’m not in
the hot seat.
Anyway, this column is where
I give away all my secrets. I’ll
tell you exactly what I look for
in a number of technical disci-
plines, and you can be well
prepared should I ever show
up. Why am I doing this? Isn’t
it typical for an assessor to
have a few ‘stumpers’ in a
back pocket just to establish
Spring 2001/Winter 2000
Position Available
Quality Technician (Senior Technical Aide)
The successful candidate will be inspecting product,
parts, tooling and supplies to insure conformance to
quality standards. May use a variety of inspection/
test equipment including but not limited to: height
gages, micrometers, OGP Video inspection system,
Sheffield coordinate measuring machine, etc.
The candidate must have excellent written and
oral communication skills, a demonstrated ability to
work in a team environment, and be able to work
overtime as the job requires.
The candidate is expected to maintain current
knowledge of quality assurance methods and incor-
porate it into our programs as appropriate. Proven
dimensional metrology expertise and proficiency in
GD&T needed. ASQ certification as a Quality
Tecnician (CQT) or Mechanical Inspector (CMI) is
preferred.
Respond to: Ted Beale
Fermi National Accelerator Laboratory
P.O. Box 500, MS 316
Batavia, Illinois 60510
V 630-840-4396
F 630-840-4653
Email beale@fnal.gov
that he or she knows more than the lab management
or technicians?
It certainly seems that way sometimes, but really what
I am looking for is a certain level of knowledge about
measurements and metrology, and if it’s there, I’ll be
happy. Most of these are little items – to some they
have even seemed trivial, and it can be annoying to get
‘dinged’ for missing some little details.
Sorry Charlie – metrology is all about the little details.
Making an unsophisticated measurement is very easy,
even for a young student or an untrained clerk. People
weigh themselves, read the outdoor temperature from
a thermometer, buy and pay for gasoline, and dispense
a cup of flour every day, and to the degree that they
need to, they get it right almost every time.
Of course metrology and calibration are usually done
to a higher degree of refinement, which almost always
 Spring 2001/Winter 2000 The STANDARD
means a smaller uncertainty. When you need to get the
‘right’ answer, and you need to get a few digits of
significance, there are always little corrections that
need to be made to the general-purpose answer you
get from reading the dial on your instruments. Knowing
about these, when they’re needed, how to calculate
them, and how to apply them, is what differentiates a
metrologist from any other scientist, engineer, or tech-
nician.
Here’s a quick list of some of these details that I will
always look for during a calibration assessment. I’m not
giving away anything – if you know how to do all of
these you are doing your job well and there’s no reason
that I should complain – I’ll be happy.
Electrical – DC/LF: If you use everyday connecting
leads (the best, and most common are made by
Pomona), you may not even have noticed that the
banana plugs on the end are chrome plated. Under-
neath is probably some sort of brass and/or nickel
alloy. The binding posts and banana jacks on all high-
precision voltmeters are made of Tellurium Copper.
When you plug up a circuit using these components,
the dissimilar metals in the connection will make a
thermocouple. If there is any temperature difference
across it at all, a Seebeck voltage will be generated that
adds to the voltage being generated or measured by
the circuit – an error. I will typically demonstrate by
holding (and warming up) one of the plugs by its handle
and this will generate as much as 10 microvolts of
offset. To make me happy, use soft copper leads with
Copper or Tellurium Copper connectors.
Measuring low resistance (up to 10kOhms, at least) to
high precision requires a four-wire connection. Resis-
tance measurement is usually accomplished by pass-
ing a current through the unknown resistor and reading
the voltage across it. In a four-wire circuit, the measur-
ing current is carried to the unit under test on different
wires than those used to read the voltage. As a result,
voltage drop in the current leads does not contribution
to the uncertainty – otherwise it would. I expect you to
know how to hook up a four-wire measurement and to
be able to explain why it is done and how it works.
The highest precision resistors are kept at a constant
temperature in a heated oil bath. The oil must be
characterized as one having an extremely high resis-
tivity or it will appear in parallel with the standard
resistor in the bath and change its apparent value.
Purchase of the oil must be by specification the in-
cludes resistivity, and incoming purchase of oil must be
checked for conformance to those specifications.
Measuring high resistance (1 GigaOhm and higher)
offers its own tricks. The problem here is that every-
thing in the measurement setup contributes little leak-
age paths (resistors in parallel with the unit under test).
Page 7
These were there at lower resistances, of course, but
contributed insignificant effects. First, you will need to
use special test leads again. The best is a lead set
specially characterized for high voltage or high resis-
tance. These wires are shielded (to prevent electro-
static noise) and Teflon® insulated. Teflon has a very
high insulation resistance and will contribute a great
deal less leakage. Cleanliness is important, too. Any
dirt or carbon path between connections will also be
part of the circuit. Clean contacts, terminal posts, and
any mounting plate (usually plastic) with a non-residue
cleaner such as Ethyl alcohol. Higher resistance re-
quires a guarding circuit. In this setup, a conductor set
to the same potential as the measuring circuit sur-
rounds the wires and connectors (and usually the unit
under test itself). This will prevent any leakage current
from flowing because there will be no potential differ-
ence to drive it. Even higher resistance (over 1
TeraOhm) should be done inside a Faraday cage to
prevent accumulation of stray static charges anywhere
in the measuring circuit.
A technician or laboratory that expects to be accredited
in these areas needs to be able to speak knowledge-
ably about these measurement issues and the under-
lying Ohm’s law theory.
Any precision measurement at a low AC frequency
should be done with shielded coaxial leads to minimize
external electrostatic, voltage, or RF interference. Twist
cables together to prevent making an open loop that
can intercept magnetic field lines and cause errors due
to induced currents. Be extra careful when making
measurements at a multiple of the power line fre-
quency, as interference there is intense and hard to get
away from.
Well, I’ve about used up my space here and I still have
many pages to go. Look in this column in the future to
see how to make your assessor happy in other metrol-
ogy disciplines and parameters.
Philip Stein is a metrology and quality consultant in
private practice in Pennington, NJ. He holds a master’s
degree in measurement science from The George
Washington University, in Washington DC, and is an
ASQ Fellow. For more information, go to
www.measurement.com.
Philip Stein, A2LA Lead Assessor, is a Past
Chair of the MQD, a past member of the Board
of Directors of ASQ, and is an ASQ Fellow
 Page 8 The STANDARD
The Learning Curve
This is the twenty-eighth in a contiguous series of often
rambling discourses, ostensibly directed towards the
subject of Metrology Education,
but frequently digressing into other
areas of interest to this author. As
in the past this one will be in the
form of an open letter to the Execu-
tive Editor, our new boss, who as
yet, has not seen fit to shut us
down.
Phil Painchaud
Dear New Boss:
It is now nearly the end of January, and the Mea-
surement Science Conference has been over for a
week. If you have never been to one and you are in the
measurements area, you should try. It has become the
largest and most prestigious annual gathering of indi-
vidual Metrology Professionals anywhere. —But
enough of the “plug” for that conclave—As I have been
for the past thirty-one years. I arrived there at the
Disneyland Hotel (I live only 11.3 miles away) early
enough to help setup the booth our Division usually has
to attract new members. Herb O’Neil, who is head
honcho for that Metrology training program up Minne-
sota way, came by where we were working with several
of his students and attempted to introduce me as “The
Man Who Invented Metrology”!
Thanks a lot, Herb! I realize that lots of folks con-
sider me ancient, archaic, and obsolete, but really I am
not that old! I have done many things during my life, a
few not so memorable. I believe that when mankind
first ceased being nomadic hunters, and became agrar-
ian farmers, it became necessary for him to divide and
measure the land to be tilled, thus making Geodesy the
oldest of the Metrological disciplines. I wasn’t around
then. I came by a few years later. I do occasionally get
some arguments from those who claim that the first
Metrologists were the nomads themselves who be-
came astronomers of sorts as they studied the skies at
night. I believe otherwise, as I feel their “star gazing”
was nearly all qualitative rather than involving those
quantitative measurements that are the hallmark of
metrology. Herb, I appreciate the complement, but
while I may be guilty of many things, and as I said, some
quite forgettable, I most certainly did not invent Metrol-
Spring 2001/Winter 2000
ogy—I am not quite old enough to have.
That brings me to another misapplied credit. A few
months ago the Editor of a sister newsletter authored
a short piece in which he credited Jerry Hayes (Jerry
was the head of the U.S. Navy Metrology Engineering
Center for nearly 30 years) with inventing the term
“Metrology.” I wrote him (the Editor) a chiding letter,
pointing out that the word “Metrology” actually comes
from the ancient Greek; “metron”, meaning “to mea-
sure,” and “ology,” meaning “the study of.” (Now don’t
go crawling all over me if my syntax is not perfect; I
make no claims of being a scholar of ancient Greek. I
was not a contemporary of Socrates.) The earliest
documentary record I possess or am aware of concern-
ing the use of the term “Metrology” is in Lord Kelvin’s
famous speech of 1886: —
—“THAT WHEN YOU CAN MEASURE WHAT YOU
ARE SPEAKING ABOUT, AND EXPRESS IT IN
NUMBERS, YOU KNOW SOMETHING ABOUT IT.
BUT, WHEN YOU CANNOT MEASURE IT, WHEN
YOU CANNOT EXPRESS IT IN NUMBERS, YOUR
KNOWLEDGE IS OF A MEAGER AND UNSATIS-
FACTORY KIND; IT MAY BE THE BEGINNING OF
KNOWLEDGE, BUT YOU HAVE SCARCELY, IN
YOUR THOUGHTS, ADVANCED TO THE STAGE
OF SCIENCE, WHATEVER THE MATTER MAY
BE. SO THEREFORE, IF SCIENCE IS MEASURE-
MENT, THEN WITHOUT METROLOGY THERE
CAN BE NO SCIENCE.”
—William Thompson—(Lord Kelvin, Baron of
Lars)—May 6. 1886—
If you think you have heard that quotation too often,
I will not apologize; memorize it—it expresses the
fundamental precept of our profession. Particularly
remember that last sentence.
I have received a number of responses concerning
my statements in the last LEARNING CURVE column
about my use of the Common Gender and its seem-
ingly male pronouns. There were a couple of brickbats,
but the general response was overwhelmingly positive.
Our Editor made some comments (not critical) to me.
I answered him as follows: “Anyone who feels that the
hierarchy above them is restricting them from rising to
their highest level of capability because of gender bias
developed as a result of the proper use of the English
Language has idiots for bosses. Total competency can
defeat any idiot no matter how well entrenched. To be
a true Metrologist, one must of necessity be totally
competent. — A second point that must be taken into
account is that I was in that column, quoting an earlier
published document. Properly documented accredita-
tion of quotations requires that you quote the original
precisely and without ‘politically correct’ alteration.” I
refuse to alter any quotation, mine or anyone else’s,
 Spring 2001/Winter 2000 The STANDARD
because of so- called “politically correct” expediencies.
In the last issue, I threatened to continue my tutori-
als drawn from a lecture I gave a number of years ago,
providing enough of you did not petition our Editor to
silence me. Either you have not beaten him down, or he
has been too busy to quiet me. Here goes. Let’s, as I
promised, talk about the METROLOGICAL ENGINEER-
ING AND ANALYSIS SUPPORT HEAD.
In my organization the Metrological Engineering
and Analysis Support Head was the individual who
headed the sub-organization that was responsible to
supply all technical support to the rest of the Metrology
Organization as well as to the rest of the facility. It might
be said that unit was our (the Metrology branch)
technology center.
• It was responsible for the acquisition, ap-
proval, testing, and writing or modifying if
necessary of all calibration procedures and
all maintenance procedures used throughout
the Metrology branch.
• It was responsible for the technical monitor-
ing of all calibrations and the analysis of all
of the data derived from each those calibra-
tions.
• From these analyses it was responsible to
develop algorithms such as would result in
the adjustment of recall periods; the correc-
tion of calibration procedures; and revision
of maintenance procedures.
• It was responsible for determining the need
for, the design of, and overseeing the con-
struction of all special calibration fixtures and
assemblies,
• It was responsible for the technical evalua-
tion of all requests for the acquisition of all
new or additional measuring devices within
the entire facility.
• It was responsible to be the technical con-
sultant on all measurement and instrumen-
tation problems throughout the facility (and
on occasion to other divisions and facilities
of the corporation).
That is quite an order! It is obvious that the individual
heading that unit needs to possess qualifications well
above average. The description we used for this posi-
tion was as follows: (Again I quote directly from the
before-referenced lecture and indirectly from the
Northrop-Nortronics Metrology Manual.)
“Metrological Engineering and Analysis Support
Head: Here we have a spot for a meteorologically
experienced professional who is more attuned to the
analytical and development approaches rather than to
hands-on-the-hardware calibrations or to standards
maintenance. As in the other supervisory positions,
Page 9
supervisory potential is a must. The minimum require-
ments for this individual are much more manifold than
are for the other Metrology activity heads because of
the broader range of responsibilities for which this
organization is accountable. In fact, the total qualifica-
tions for a proper individual to optimally fill this position
are far too lengthy to completely delineate here. But,
since this individual will be leading a high technology
organization, he must be able and willing, as well as
qualified both technically and psychologically, to as-
sume the responsibility, i.e. to assume the responsibil-
ity for the general directions or methods to be followed.
1. He must be able to wisely and rapidly evaluate
proposed methods and directions in terms of cor-
rectness, validity, rapidity convenience, and reli-
ability.
He must always view each of these factors in
respect to the following questions
• Is it in the best interests of the company?
• Is the cost reasonable, minimal, and is it also
justifiable?
• Is it proper for the Metrology organization?
• Will metrological, professional, or personal
integrity be jeopardized in any way?
2. He must be capable of assuming responsibility for
consultations, both within and outside the Metrol-
ogy organization, specifying the technical proce-
dures to be used within Metrology as well as advis-
ing on those to be used outside, while controlling the
internal validity of all tests and calibrations.
3. He must be professionally capable of recognizing
that he coordinates the organization’s technical
conscience and of displaying and exercising an
unusual depth and degree of independent sole
judgment which can only be based upon a high
degree of maturity combined with metrological and
other technical experience in depth.
4. He must be constantly striking a balance between
safety (in measurement assurance) and economy,
with full knowledge that absolute assurance (safety)
costs infinite dollars: he must be capable of direct-
ing as well as personally performing rapid and
meaningful statistical analyses, yet being fully cog-
nizant that statistical results alone can be danger-
ously misleading.
5. He must be constantly aware that good and exten-
sive technical documentation and extensive techni-
cal records are the only base upon which to erect a
structure of metrological statistical analysis, which,
in turn, provides the means of estimating the Quality
risks for which he and the Administrative Metrolo-
gist are responsible.
 Page 10 The STANDARD
6. He must realize that Metrology, as in many rapidly
advancing fields, often requires judgment born of
experience. This characteristic is sometimes prop-
erly expressed as ‘expert opinion’ and improperly
as ‘educated guesses.’ The professional heading
the Metrology Analysis and Engineering function
must assume the position of coordinator of the
metrology organization group ‘expert opinion.’ To
accomplish this, he must possess an established
reputation. The exercise of such judgment resides
in the expertise and experience of the individual
making the judgment and, more particularly, in the
group leader himself. Even if this leader has re-
markable capability, he may well be quite ineffec-
tual and un-believed unless he has also gained an
established reputation.
A parallel to this can be seen in decisions often
required of economists and other leaders in busi-
ness. Action on economic decisions must often be
taken based on an individual’s reputation rather
than on rigid proof. Quite often rigid proof is just not
readily available or timely to the problem at hand.
It is not too difficult to see that the Metrological
Engineering and Support Head (or what-ever you are
about to call him in your organization) must be an
individual of character, stature, profound knowledge,
and of absolute integrity, not just some jerk who has
Certified Calibration Technician (CCT) Program Update
ASQ milestones that must be passed in the estab-
lishment of a new certification program, or what might
be construed as challenges in the quest for certifica-
tion, are varied in both scope and complexity. Reach-
ing any one of these milestones requires the resources
of many talented individuals who believe certification is
not only warranted but also highly desirable for an
industry/discipline. Policy changes by the ASQ Certifi-
cation Board have made creation of new industry/
discipline specific certifications possible. ASQ has
created a procedure for establishing new certification
programs that clearly defines program milestones.
This procedure reflects the corporate wisdom and
experience ASQ has gained over the years via its
existing certification programs i.e. CQE, CRE, etc. The
procedure is written so as to preclude reinvention of the
wheel and provides guidance in order to avoid known
potholes. Within this procedure checks are made to
ensure potential markets are adequate to sustain the
program, the certification’s body of knowledge is unique,
attainable and comprehensive, and commitment for
Spring 2001/Winter 2000
been yanked off of the bench because he couldn’t hack
it there and has favorably caught the attention of
someone in the hierarchy. This individual along with
the Administrative Metrologist constitutes the two most
important people in your entire Metrology system.
They are the heart, soul, and brains of the entire
operation.
I still have a few copies of the handout from that
lecture. They may have to be dusted off, but they are
still readable. I you would like one, please write to me
and I shall be happy to send one to you. If you want to
argue, I’m your man, please contact me.
By next issue we should be able to update you
concerning the Four-Year Degree Program for a Bach-
elor of Science in Quality Assurance (with an Option in
Measurement Science) which is in gestation at the
California State University—Dominguez Hills.
Phil Painchaud
1110 West Dorothy Drive
Brea, CA 92821-2017
Phone: (714) 529-6604
Fax: (714) 529-1109
e-Mail: 72320.134@compuserve.com or:
olepappy@JUNO,com
support of the program is evident before the real work
of certification begins. ASQ personnel involved in
overseeing the certification process are extremely
knowledgeable and provide assistance, insight and
wisdom on an ‘as needed’ basis. Committee activities
continue within this comprehensive framework in a
concerted effort to establish a certification program for
calibration technicians, commonly referred to as the
proposed Certified Calibration Technician program or
simply CCT program.
Certification for calibration technicians has been a
heated topic for debate for many years in Metrology
circles. General consensus has been that it is only a
matter of time before some professional organization
succeeds in establishment of such a program. ASQ’s
Measurement Quality Division has committed to be
that organization. The following is a brief description, in
chronological order, of events required to establish an
ASQ certification program:
 Spring 2001/Winter 2000 The STANDARD
1. Prepare a written Initial proposal justifying the es-
tablishment of the new certification. The written
Initial proposal shall address:
a. Uniqueness within the quality profession
b. Applicability within industry
c. Public domain breadth of the body of
knowledge
d. Consistency with ASQ’s objectives, policies
and procedures
e. Training availability
f. Support and commitment
g. Market needs
2. Presentation and justification of the written Initial
proposal to ASQ’s Certification Board. The board
votes whether or not to proceed with the program
based on information provided.
3. Perform a job analysis under the auspices of a
psychometrician. This typically is a contracted ac-
tivity requiring substantial funding commitment. The
job analysis focuses on the work of 10-12 subject
matter experts (SME) who are tasked in helping
develop questions for a job analysis survey. The
results of this survey, as completed by calibration
practitioners, is compiled and summarized.
4. Prepare written Final proposal based on job analy-
sis. The written final proposal shall include the
following:
a. Program narrative and summary
b. Expenditure and implementation plan
c. Financial and market analysis
d. Supporting elements and job analysis results
5. Presentation and justification of the written Final
proposal to ASQ’s Certification Board. The board
votes whether or not to proceed with the program
based on information provided.
6. Presentation and justification of the written Final
proposal to ASQ’s Professional Development Coun-
cil (PDC). The council votes whether or not to
proceed with the program based on information
provided.
7. Presentation and justification of the written Final
proposal to ASQ’s Board of Directors. The board
votes whether or not to proceed with the program
based on information provided. Upon approval the
certification becomes an official ASQ certification.
My first reading of ASQ’s certification procedure left
me with a rather ‘uphill climb’ feeling in my stomach
given the Initial and Final proposal approvals that must
be obtained from the Certification Board before the
PDC and Board of Directors can be approached for
their consideration. This feeling soon left me as I
Page 11
realized the process of obtaining two separate approv-
als from the Certification Board would have the desir-
able effect of ‘fine-tuning’ the CCT program proposal,
thereby giving it the best possible chance of winning
approval at the higher decision levels.
Upon querying a psychometrician (ASQ suggested
contractor) regarding job analysis activities, referenced
in ASQ’s certification procedure, I learned the steps
proven successful in other ASQ certifications. Typi-
cally, the first step in performing a job analysis is to
identify and review available background information.
Work related background information regarding cali-
bration technicians has been identified from various
sources and was compiled in some detail in the Initial
CCT proposal. This background information will be
used in developing an interview script. The interview
script will be the basis for phone interviews with at least
10 subject matter experts (SME) identified by the CCT
committee. The main reason telephone interviews are
performed is to collect as much background informa-
tion as possible prior to conducting a job analysis
workshop (considered the key in achieving a useful job
analysis). The psychometrician explained to me that
telephone interviews are a time proven way of collect-
ing job related information from a perspective (individu-
ally) other than that of a job analysis workshop (collec-
tively). The data collected from the phone interviews
are compiled, summarized and disseminated to SMEs
prior to a job analysis workshop. This has the advan-
tage of assuring SMEs, who will be participating in a job
analysis workshop, will start on the same page and
provides a good means for coverage of information
that would normally take time away from other activi-
ties. SMEs that contribute via phone interviews are not
normally the same SMEs participating in the job analy-
sis workshop, thus providing opportunity for participa-
tion by those SMEs who want to contribute but may not
be able to attend a job analysis workshop.
A job analysis workshop is normally a one-and-a-
half to two-day session involving 10-12 SMEs. Activi-
ties are coordinated and coached by a psychometri-
cian who guides attendees in reviewing existing ma-
terials, telephone interview results and other pertinent
information. The calibration technician job analysis
workshop will focus on drafting a delineation of do-
mains, tasks and knowledge pertinent to calibration
technicians. After the job analysis workshop, the draft
delineation is reviewed and finalized via a mail review
process with workshop attendees. The final delinea-
tion is the basis for development of a validation survey.
As one would expect, the validation survey is an
instrument comprised of instructions, weighted scales,
demographic questions and other open-ended ques-
tions. This step, probably more than any other step in
the job analysis process, requires the guidance of a
 Page 12 The STANDARD
practicing psychometrician to insure questions are not
only grammatically correct but also are not vague or
misleading to the intended audience. Common rating
scales are used in validation surveys, regarding do-
mains, tasks and knowledge, dealing with criticality
and time / frequency. The validation survey undergoes
several reviews before the final revision is printed and
mailed. The actual number of surveys mailed is deter-
mined by a sampling scheme reflecting the number of
estimated practitioners.
Validation survey results are compiled and summa-
rized in a report using various graphic and tabular
presentations in order to highlight and discern trends
and outline similarities and differences. This informa-
tion will be used to develop the Final CCT proposal.
The Final CCT proposal is to be presented to the
Certification board for approval where, if successful, it
will proceed to the PDC for consideration. Upon PDC
approval, the Board of Directors will make the final
decision in determining if the CCT program will become
a reality.
So where is the CCT program pursuant to these
discussed milestones as of this writing (late January
2001)?
• Written Initial CCT proposal unanimously
approved by Certification board (October
2000)
• Proposal for CCT Job Analysis Study by
Professional Examination Services
(PES)(submitted December 2000)
Inspection Division
by Navin Shamji Dedhia
The mission of the Inspection Division is to increase
customer satisfaction through continuous improvment
by disseminating information relating to the quality of
products, goods and services through inspection and
test methodology. Its vision is to be the World’s Lead-
ing Authority on issues related to inspection technolo-
gies, tools, techniques, methods, principles, and
applications geared toward improving quality and pro-
ductivity.
The Division is headed by Navin S. Dedhia as a Chair.
He has been actively involved in ASQ activities at the
Local, Division, and National Level in various capaci-
ties. He is the past recipient of the E. Jack Lancaster
Award from ASQ. He has received testimonial awards
from ASQ and been recognized by the organizations
and institutions around the world.
Jenny Persfull is the current Chair-Elect from Indiana
State. She has been actively involved in the Local
Spring 2001/Winter 2000
• Funding approved for Job Analysis Survey
by PES (January 2001)
• Funding approved for Final CCT proposal
expenses (January 2001)
• Job Analysis workshop scheduled for dates
following NCSL national conference (1,2
August 2001)
• Validation survey final report scheduled for
December 2001
The timetables for the remainder of certification
milestones i.e. Final CCT presentations will be final-
ized in January 2002.
The CCT program has progressed to its present
state through the hard work of many individuals who
believe a program for certifying calibration technicians
is needed for many of the same reasons other ASQ
certification programs were established. For those
individuals wishing to contribute to this important initia-
tive or who may have questions / suggestions concern-
ing the CCT program please contact the CCT chair at:
Chris.Grachanen@Compaq.Com
ASQ’s Measurement Quality Division is proud to be the
divisional sponsor for the ASQ’s CCT program.
If interested please E-mail MQD’s committee chair
at: chris.grachanen@compaq.com.
Chris Grachanen
ASQ MQD Certification Committee Chair
Section, Certification Improvment Workshops, and Di-
vision Affairs.
Greg Gay is the current Secretary of the Division from
Michigan State. He has been actively involved in the
Certified Mechanical Inspection Examination process
and workshop. He is also past recipient of the Inspector
of the Year Award.
Bill Sherman is the current Treasurer and past Chair of
the Division. Bill has been actively involved in the
Division Affairs for many years.
Other Council members, who are actively involved
include Jim Cooper as the past Chair and current
Newsletter Editor, Chuck Carter as the past Chair, Joy
Flynn as the Vice Chair, Mollie Brown, Bruce Johnson
as the past Chair, Clarence Johnson, Lou Samudio,
Charles Leach, Bill Prince and Bud Gookins.
 Spring 2001/Winter 2000 The STANDARD Page 13

12 Steps to Useful Software Metrics

Linda Westfall
The Westfall Team
westfall@idt.net
PMB 383, 3000 Custer Road, Suite 270
Plano, TX 75075
972-867-1172 (voice)
972-943-1484 (fax)

Abstract: 12 Steps to Useful Software Metrics introduces the reader to a practical process for establishing and
tailoring a software metrics program that focuses on goals and information needs. The process provides a
practical, systematic, start-to-finish method of selecting, designing and implementing software metrics. It outlines
a cookbook method that the reader can use to simplify the journey from software metrics in concept to delivered
information.
Bio: Linda Westfall is the President of The Westfall Team, which provides Software Metrics and Software Quality
Engineering training and consulting services. Prior to starting her own business, Linda was the Senior Manager
of the Quality Metrics and Analysis at DSC Communications where her team designed and implemented a
corporate-wide metric program. Linda has twenty years of experience in real-time software engineering, quality
and metrics. She has worked as a Software Engineer, Systems Analyst, Software Process Engineer and Manager
of Production Software.
Very active professionally, Linda Westfall is Chair Elect of the American Society for Quality (ASQ) Software
Division. She has also served as the Software Division’s Program Chair and Certification Chair and has served
on the ASQ National Certification Board. Linda wrote the Software Metrics and Software Project Management
sections of the ASQ Software Quality Engineering course and co-authored the ASQ Software Metrics course.
Key Words/Phrases: Software Metrics, Software Measurement, Data Collection

12 Steps to Useful Software Metrics

What Are Software Metrics?
Software metrics are an integral part of the state-of- Measurement
the-practice in software engineering. Taking an activ-
Based
What Are
ity-based view, Goodman defines software metrics as Software Metrics?
Techniques
“The continuous application of measurement-based
techniques to the software development process and
Applied
its products to supply meaningful and timely manage- To
ment information, together with the use of those tech-
niques to improve that process and its products.” To
[Goodman-93] Figure 1 illustrates an expansion of this Processes, Supply Engineering &
definition to include software-related services such as Products & Management
installation and responding to customer issues. Soft- Services To Information
Improve
ware metrics can provide the information needed by
engineers for technical decisions as well as informa-
tion required by management. Taking a measurement-
based view, Schulmeyer defines a metric as “a quan- Figure 1: What Are Software Metrics?
titative measure of the degree to which a system, component or process possesses a given attribute. [Schulmeyer-
98]
If a metric is to provide useful information, everyone involved in selecting, designing, implementing, collecting, and
utilizing it must understand its definition and purpose. This paper outlines twelve steps to selecting, designing and
implementing software metrics in order to insure this understanding.
 Page 14 The STANDARD Spring 2001/Winter 2000

Some Basic Measurement Theory

The use of measurement is common. We use mea- Entity:
surements in everyday life to do such things as Attribute:
weigh ourselves in the morning or when we check Features &
the time of day or the distance we have traveled in Properties
our car. Measurements are used extensively in most
areas of production and manufacturing to estimate
costs, calibrate equipment, measure quality, and
monitor inventories. Science and engineering disci-
plines depend on the rigor that measurements pro-
Mapping:
vide, but what does measurement really mean? Numbers
According to Fenton, “measurement is the process
&
by which numbers or symbols are assigned to at- Symbols
tributes of entities in the real world in such a way as
to describe them according to clearly defined rules” Figure 2: Measurement Defined
[Fenton-91]. An entity is a person, place, thing, event or time period. An attribute is a feature or property of the
entity.
To measure, we must first determine the entity. For example, we could select a car as our entity. Once we select
an entity, we must select the attribute of that entity that we want to describe. For example, the car’s speed or the
pressure in its tires would be two attributes of a car. Finally, we must have a defined and accepted mapping system.
It is meaningless to say that the car’s speed is 65 or its tire pressure is 75 unless we know that we are talking about
miles per hour and pounds per square inch, respectively.
We will use the basic process model of input - process - output to discuss software entities. Software entities of
the input type include all of the resources used for software research, development, and production. Examples
of input entities include people, materials, tools, and methods. Software entities of the process type include
software-related activities and events and are usually associated with a time factor. Examples of process entities
include defined activities such as developing a software system from requirements through delivery to the
customer, the inspection of a piece of code, or the first 6 months of operations after delivery. Process entities also
include time periods, which do not necessarily correspond to specific activities. An example would be the period
between 1/1/93 and 2/1/93. Software entities of the output type are the products of the software process. These
include all the artifacts, deliverables, and documents that are produced. Examples of software output entities
include requirements documentation, design specifications, code (source, object & executable), test documen-
tation (plans, scripts, specifications, cases, reports), project plans, status reports, budgets, problem reports, and
software metrics.
Each of these software entities has many properties or features that we might want to measure. We might want
to examine a computer's price, performance, or usability. We could looks at the time or effort that it took to execute
a process, the number of incidents that occurred during the process, its cost, controllability, stability, or
effectiveness. We might want to measure the complexity, size, modularity, testability, usability, reliability, or
maintainability of a piece of source code.
One of the challenges of software metrics is that few standardized mapping systems exist. Even for seemingly
simple metrics like the number of lines of code, no standard counting method has been widely accepted. Do we
count physical or logical lines of code? Do we count comments or data definition statements? Do we expand
macros before counting and do we count the lines in those macros more than once? Another example is
engineering hours for a project – besides the effort of software engineers, do we include the effort of testers,
managers, secretaries, and other support personnel? A few metrics which do have standardized counting criteria
include McCabe’s Cyclomatic Complexity and the Function Point Counting Standard from the International
Function Point User Group (IFPUG). However, the selection, definition, and consistent use of a mapping system
within the organization for each selected metric are critical to a successful metrics program.
Introduction to the Twelve Steps
Based on all of the possible software entities and all the possible attributes of each of those entities, there are
multitudes of possible software metrics. So how do we pick the metrics that are right for our organizations? The
first four steps defined in this paper will illustrate how to identify metrics customers and then utilize the goal/
question/metric paradigm to select the software metrics that match the information needs of those customers.
 Spring 2001/Winter 2000 The STANDARD Page 15

Steps 5-10 present the process of designing and tailoring the selected metrics, including definitions, models,
counting criteria, benchmarks and objectives, reporting mechanisms, and additional qualifiers. The last two steps
deal with implementation issues, including data collection and the minimization of the impact of human factors on
metrics.
Step 1 – Identify Metrics Customers
The customers for a metrics are the people who will be making decisions or taking action based upon the metrics
– that is, the people who need the information supplied by the metrics.
There are many different types of customers for a metrics program. This adds complexity to the program because
each customer type may have different information requirements. Customers may include:
• Functional Management - interested in the ability to apply greater control to the software
development process, reducing risk and maximizing return on investment.
• Project Management - interested in controlling the projects they are in charge of and commu-
nicating facts to their management.
• Software Practitioners - the people that actually do the software development. They are also
responsible for collecting a significant amount of the data required for the metrics program.
• Specialists - including individuals performing specialized functions. (e.g., Marketing, Quality
Assurance, Process Engineering, Configuration Management, Testing, Audits, and Assess-
ments, Customer Technical Assistance).
• Users - interested in on-time delivery of high-quality software products.
If a metric does not have a customer, it should not be produced. Metrics are expensive to collect, report, and
analyze; so, if no one is using a metric, producing it is a waste of time and money.
The customers’ information requirements should always drive the metrics program. Otherwise, we may end up
with a product without a market and with a program that wastes time and money. By recognizing potential
customers and involving those customers early in the metric definition effort, the chances of success are greatly
increased.
When talking to our customers, we may find many of their individual needs are related to the same goal or problem
but expressed from their perspective or in the terminology of their specialty. Many times, what we hear is their
frustrations.
For example, the Project Manager may need to improve the way project schedules are estimated. The Functional
Manager is worried about late deliveries. The practitioners complain about overtime and not having enough time
to do things correctly. The Test Manager states that by the time the test group gets the software it’s too late to test
it completely before shipment.
When selecting metrics, we need to listen to these customers and, where possible, consolidate their various goals
or problems into statements that will help define the metrics that are needed by our organization or team.
In our example, all these individuals are asking for an improved and realistic schedule estimation process.
Step 2 – Target Goals
Basili and Rombach [Basili-88] define a Goal/
Goal / Question / Metric
Question/Metric paradigm that provides an ex-
cellent mechanism for defining a goal-based Goal 2
Goal 1
measurement program. Figure 3 illustrates the
Goal/Question/Metric paradigm.
The second step in setting up a metrics program
Question 1 Question 2 Question 3 Question 4
is to select one or more measurable goals. These
may be high-level strategic goals (e.g., minimiz-
ing cost or maximizing customer satisfaction).
Metrics 1 Metric 2 Metric 3 Metric 4 Metric 5
They may also be specific goals (e.g., evaluating
a new design method or determining whether a
[Basili-88]
product is ready to be shipped).
Figure 3: Goal/Question/Metric Paradigm
 Page 16 The STANDARD Spring 2001/Winter 2000

Software metrics programs must be designed to provide the specific information necessary to manage software
projects and improve software engineering processes and services. Organizational, project, and task goals are
determined in advance; and then metrics are selected based on those goals. The metrics are used to determine
our effectiveness in meeting these goals.
The foundation of this approach is asking not so much “what should I measure?” but “why am I measuring?” or
“what business needs does the organization wish its measurement initiative to address?” [Goodman-93]
Measuring software is a powerful way to track progress towards project goals. As Grady states, “Without such
measures for managing software, it is difficult for any organization to understand whether it is successful, and it
is difficult to resist frequent changes of strategy” [Grady-92]. Measurements help both management and engineers
maintain their focus on their goals.
Step 3 – Ask Questions
The third step is to define the questions that need to be answered in order to ensure that each goal is being
obtained. For example, if our goal was to ship only defect-free software, we might select the questions:
• Is the software product adequately tested?
• How many defects are still undetected?
• Are all known defects corrected?
Step 4 – Select Metrics
The fourth step is to select metrics that provide the information needed to answer these questions. Each selected
metric now has a clear objective – to answer one or more of the questions that need to be answered to determine
if we are meeting our goals.
When we are selecting metrics, we must be practical, realistic, and pragmatic. Avoid the “ivory-tower” perspective
that is completely removed from the existing software-engineering environment. Start with what is possible within
the current process. Once we have a few successes, our customers will be open to more radical ideas – and may
even come up with a few of their own.
Also, remember software metrics don't solve problems. People solve problems. Software metrics act as indicators
and provide information so people can make more informed decisions and intelligent choices.
An individual metric performs one of four functions. Metrics can help us Understand more about our software
products, processes, and services. Metrics can be used to Evaluate our software products, processes, and
services against established standards and goals. Metrics can provide the information we need to Control
resources and processes used to produce our software. Metrics can be used to Predict attributes of software
entities in the future. [Humphrey-89]. A comprehensive Software Metric program would include metrics that
perform all of these functions.
The objective for each metric can be formally defined in terms of one of these functions: the attribute of the entity
being measured and the goal for the measurement. This leads to the following metrics objective template:

understand
attribute
evaluate in order
To the of the goal(s)
control to
entity
predict

An example of the use of this template for the “percentage of known defects corrected: metric would be:
% known all known
defects defects are
in order
To evaluate the corrected corrected
to
during before
development shipment

Having a clearly defined and documented objective statement for each metric has the following benefits:
• Provides a rigor and discipline that helps ensure a well-defined metric based on customer
goals.
• Eliminates misunderstandings about how the metric is intended to be used.
 Spring 2001/Winter 2000 The STANDARD Page 17

• Communicates the need for the metric, which can help in obtaining resources to implement the
data collection and reporting mechanisms.
• Provides the base requirements statement for the design of the metric.
Step 5 – Standardize Definitions
The fifth step is to agree to a standard definition for the entities and their measured attributes. When we use terms
like defect, problem report, size, and even project, other people will interpret these words in their own context with
meanings that may differ from our intended definition. These interpretation differences increase when more
ambiguous terms like quality, maintainability, and user friendliness are used.
Additionally, individuals may use different terms to mean the same thing. For example, the terms defect report,
problem report, incident report, fault report, or customer call report may be used by various organizations to mean
the same thing, but unfortunately they may also refer to different entities. One external customer may use customer
call report to refer to their complaint and problem report as the description of the defect in the software, while
another customer may use problem report for the initial complaint. Differing interpretations of terminology may be
one of the biggest barriers to understanding.
Unfortunately, there is little standardization in the industry of the definitions for most software attributes. Everyone
has an opinion and the debate will probably continue for many years. Our metrics program cannot wait that long.
The approach I suggest is to adopt standard definitions within your organization and then apply them consistently.
You can use those definitions that exist within the industry as a foundation to get you started, for example,
definitions from the IEEE Glossary [IEEE-83] or those found in software engineering and metrics literature. Pick
the definitions that match your organizational objectives or use them as a basis for creating your own definition.
Step 6 – Choose a Model
The sixth step is to derive a model for the metric. In simple terms, the model defines how we are going to calculate
the metric. Some metrics, called metric primitives, are measured directly; and their model typically consists of a
single variable. Other more complex metrics are modeled using mathematical combinations of metrics primitives
or other complex metrics.
All modeling includes an element of simplification. This is both the strength and the weakness of modeling. When
we create a software measurement model, we need to be pragmatic. If we try to include all of the elements that
affect the attribute or characterize the entity, our model can become so complicated that it’s useless. Being
pragmatic means not trying to create the most comprehensive model. It means picking the aspects that are the
most important. Remember that the model can always be modified to include additional levels of detail in the future.
Ask yourself the questions:
• Does the model provide more information than we have now?
• Is the information of practical benefit?
• Does it tell us what we want to know?
There are two methods for selecting a model: use an existing model or create a new one. In many cases, there
is no need to “reinvent the wheel.” Many software metrics models exist that have been used successfully by other
organizations. These are documented in the current literature and in proprietary products that can be purchased.
With a little research, we can utilize these models with little or no adaptation to match our own environment.
The second method is to create our own model. The best advice here is to talk to the people who are actually
responsible for the product or resource or who are involved in the process. They are the experts. They know what
factors are important. If we create a new model for our metric, we must ensure the model is intelligible to our
customers, and we must prove it is a valid model for what we are trying to measure. Often, this validation can occur
only through application of statistical techniques.
To illustrate the selection of a model, let’s consider a metric for the duration of unplanned system outages. If we
are evaluating a software system installed at a single site, a simple model such as minutes of outage per calendar
month may be sufficient. If our objective is to compare different software releases installed on varying numbers
of sites, we might select a model such as minutes of outage per 100 operation months. If we wanted to focus in
on the impact to our customers, we might select minutes of outage per site per year.
Step 7 – Establish Counting Criteria
The seventh step in designing a metric is to break the model down into its lowest level metric primitives and define
 Page 18 The STANDARD Spring 2001/Winter 2000

the counting criteria used to measure each primitive. This defines the mapping system for the measurement of
each metric primitive.
The importance of the need for defining counting criteria can be illustrated by considering the lines of code metric.
Lines of code is one of the most used and most often misused of all of the software metrics. The problems,
variations, and anomalies of using lines of code are well documented [Jones-86], and there is no industry-accepted
standard for counting lines of code. Therefore, if you are going to use a metric based on lines of code, it is critical
that specific counting criteria be defined. These criteria must also accompany the metric in all reports and analysis
so that metrics customers can understand the definition of the metric. Without this, invalid comparisons with other
data are almost inevitable.
The metric primitives and their counting criteria define the first level of data that needs to be collected in order to
implement the metric. To illustrate this, let’s use the model of minutes of system outage per site per year. One of
the metrics primitives for this model is the number of sites. At first, counting this primitive seems simple. However,
when we consider the dynamics of adding new sites or installing new software on existing sites, the counting
criteria become more complex. Do we use the number of sites on the last day of the period or calculate some
average number of sites for the period? Either way, we will need to collect data on the date the system was installed
on the site. In addition, if we intend to compare different releases of the software we will need to collect data on
what releases have been installed on each site and when each was installed.
Step 8 – Decide What’s Good
The eighth step in designing a metric is defining “what's good”. Once you have decided what to measure and how
to measure it, you have to decide what to do with the results. Is 10 too few or 100 too many? Should the trend be
up or down? What do the metrics say about whether or not the product is ready to ship?
One of the first places to start looking for "what's good" is the customer. Many times, user requirements dictate
certain values for some metrics. There may be product reliability levels that must be met. The customer may have
a defined expectation of defect reduction from release to release or a required repair time for discovered defects.
Another source of information is the metrics literature. Research and studies have helped establish industry-
accepted norms for standard measures. For example, studies have established that modules of source code
should have a McCabe's Cyclomatic Complexity of <= 10 or they may be error-prone and difficult to maintain.
The best source of information on “what's good” is your own data. Processes vary from group to group. Many
metrics do not have industry accepted counting criteria (i.e., lines of code), and most documentation does not
include how the metrics were calculated. Therefore, comparing your values to published standards may result in
erroneous interpretations. Whenever possible, use your organization's own data to determine baselines and
establish objectives for you metrics. If historical data is not available, wait until enough data is collected to
reasonably establish current values.
Once you have decided where you are and where you want to be, you can determine whether or not action is
needed. If no action is necessary, management can either turn its attention elsewhere or establish some
monitoring actions to insure that the value stays at desired level. However, if improvements are needed, goals can
be established to help drive and monitor improvement activities. When setting metrics goals remember:
• The goal must be reasonable. It’s all right to establish a stretch goal; but if the goal is unre-
alistic, everyone will just ignore it.
• A goal must be relevant in the eyes of the people that must achieve it, or they will not
understand (and not support) it.
• The goal must be associated with a time frame. Nothing happens overnight. To say a 50%
backlog reduction without a “by when” is meaningless.
• The goal must be based on supporting actions. It may be reasonable to set a goal of 50%
backlog reduction for defects if a special team is created to concentrate on fixing problems. If
everything is “business as usual,” do not expect setting a goal to have any effect.
• Unless it is achievable in a short time, the goal must be broken down into small increments.
If the “by when” is a long way off, it is only human nature to procrastinate. If there is a year
to accomplish the improvement, it will be easy to put it on the back burner in preference to
more pressing concerns. If we divide the same goal into 12 smaller end-of-the-month goals,
actions are more likely to be taken now.
 Spring 2001/Winter 2000 The STANDARD Page 19

Step 9 – Define Reporting Mechanisms

The ninth step is to decide how to report the metric. This includes defining the report format, data extraction and
reporting cycle, reporting mechanisms, distribution, and availability.
The report format defines what the report looks like. Is the metric included in a table with other metrics values for
the period? Is it added as the latest value in a trend chart that tracks values for the metric over multiple periods?
Should that trend chart be a bar, line, or area graph? Is it better to compare values using stacked bars or a pie
chart? Do the tables and graphs stand alone, or is there detailed analysis text included with the report? Are goals
or control values included in the report?
The data extraction cycle defines how often the data snap-shot(s) are required for the metric and when they will
be available for use for calculating the metric. The reporting cycle defines how often the report is generated and
when it is due for distribution. For example, root cause analysis metrics may be triggered by some event, like the
completion of a phase in the software development process. Other metrics like the defect arrival rate may be
extracted and reported on a daily basis during system test and extracted on a monthly basis and reported quarterly
after the product is released to the field.
The reporting mechanism outlines the way that the metric is delivered (i.e., hard copy report, email, on-line
electronic data).
Defining the distribution involves determining who receives regular copies of the report or access to the metric.
The availability of the metrics defines any restrictions on access to the metric (i.e., need to know, internal use only)
and the approval mechanism for additions and deletions to access or standard distribution.
Step 10 – Determine Additional Qualifiers
The tenth step in designing a metric is determining the additional metric qualifiers. A good metric is a generic
metric. That means the metric is valid for an entire hierarchy of additional qualifiers. For example, we can talk about
the duration of unplanned outages for an entire product line, an individual product, or a specific release of that
product. We could look at outages by customer or business segment. Alternatively, we could look at them by type
or cause.
The additional qualifiers provide the demographic information needed for these various views of the metric. The
main reason additional qualifiers need to be defined as part of the metrics design is that they determine the second
level of data collection requirements. Not only is the metric primitive data required, but data also has to exist to
allow the distinction between these additional qualifiers.
Step 11 – Collect Data
The question of “what data to collect?” was actually answered in steps 7 and 10 above. The answer is to collect
all of the data required to provide the metrics primitives and the additional qualifiers.
In most cases, the “owner” of the data is the best answer to the question of “who should collect the data?” The
data “owner” is the person with direct access to the source of the data and in many cases is actually responsible
for generating the data. Table 1 illustrates the owners of various kinds of data.
Benefits of having the data owner collect the data include:
• Data is collected as it is being generated, which increases accuracy and completeness
• Data owners are more likely to be able to detect anomalies in the data as it is being collected,
which increases accuracy
• Human error caused by duplicate recording (once by data recorder and again by data entry
clerk) is eliminated, which increases accuracy
Once the people who gather the data are identified, they must agree to do the work. They must be convinced of
the importance and usefulness of collecting the data. Management has to support the program by giving these
people the time and resources required to perform data collection activities. A support staff must also be available
to answer questions and to deal with data and data collection problems and issues.
 Page 20 The STANDARD Spring 2001/Winter 2000

Owner Examples of Data Owned

Management • Schedules
• Budgets
Engineers • Time spent per task
• Inspection data including defects
• Root cause of defects
Testers • Test cases planned/executed/passed
• Problem reports from testing
• Test coverage
Configuration Management Specialists • Lines of code
• Modules changed
Users • Problem reports from operations
• Operational hours
Table 1: Examples of Data Ownership
A training program should be provided to help insure that the people collecting the data understand what to do
and when to do it. As part of the preparation for the training program, suitable procedures must be established and
documented. For simple collection mechanisms, these courses can be as short as one hour. I have found that
hands-on, interactive training, where the group works actual data collection examples, provides the best results.
Without this training, hours of support staff time can be wasted answering the same questions repeatedly. An
additional benefit of training is that it promotes a common understanding about when and how to collect the data.
This reduces the risk of collecting invalid and inconsistent data.
If the right data is not collected accurately, then the objectives of the measurement program cannot be
accomplished. Data analysis is pointless without good data. Therefore, establishing a good data collection plan
is the cornerstone of any successful metrics program. Data collection must be:
• Objective: The same person will collect the data the same way each time.
• Unambiguous: Two different people collecting the same measure for the same item will collect
the same data.
• Convenient: Data collection must be simple enough not to disrupt the working patterns of the
individual collecting the data. Therefore, data collection must become part of the process and
not an extra step performed outside of the workflow.
• Accessible: In order for data to be useful and used, easy access to the data is required. This
means that, even if the data is collected manually on forms, it must ultimately be included in
a metrics database.
There is widespread agreement that as much of the data gathering process as possible should be automated. At
a minimum, standardized forms should be used for data collection, but at some point the data from these forms
must be entered into a metrics database if it is to have any longer-term usefulness. I have found that information
that stays on forms quickly becomes buried in file draws never to see the light of day again.
Dumping raw data and hand tallying or calculating metrics is another way to introduce human error into the metrics
values. Even if the data is recorded in a simple spreadsheet, automatic sorting, data extraction, and calculation
are available and should be used. Using a spreadsheet or database also increases the speed of producing the
metrics over hand tallies.
Automating metrics reporting and delivery eliminates hours spent standing in front of copy machines. It also
increases usability because the metrics are available on the computer instead of buried in a pile of papers of the
desk. Remember, metrics are expensive. Automation can reduce the expense while making the metrics available
in a timelier manner.
Step 12 – The People Side of the Metrics Equation
No discussion on selecting, designing and implementing software metrics would be complete without a look at how
measurements affect people and people affect measurements. Whether a metric is ultimately useful to an
 Spring 2001/Winter 2000 The STANDARD Page 21

organization depends upon the attitudes of the people involved in collecting the data, calculating, reporting, and
using the metric. The simple act of measuring will affect the behavior of the individuals being measured. When
something is being measured, it is automatically assumed to have importance. People want to look good;
therefore, they want the measures to look good. When creating a metric, always decide what behaviors you want
to encourage. Then take a long look at what other behaviors might result from the use or misuse of the metric. The
best way I have found to avoid human factors problems in working with metrics is to follow some basic rules:
Don’t measure individuals: The state-of-the-art in software metrics is just not up to this yet. Individual productivity
measures are the classic example of this mistake. Remember that we often give our best people the hardest work
and then expect them to mentor others in the group. If we measure productivity in lines of code per hour, these
people may concentrate on their own work to the detriment of the team and the project. Even worse, they may come
up with unique ways of programming the same function in many extra lines of code. Focus on processes and
products, not people.
Never use metrics as a “stick”: The first time we use a metric against an individual or a group is the last time
we get valid data.
Don’t ignore the data: A sure way to kill a metrics program is to ignore the data when making decisions. “Support
your people when their reports are backed by data useful to the organization” [Grady-92]. If the goals we establish
and communicate don’t agree with our actions, then the people in our organization will perform based in our
behavior, not our goals.
Never use only one metric: Software is complex and multifaceted. A metrics program must reflect that
complexity. A balance must be maintained between cost, quality and schedule attributes to meet all of the
customer’s needs. Focusing on any one single metric can cause the attribute being measured to improve at the
expense of other attributes.
Select metrics based on goals: Metrics act as a big spotlight focusing attention on the area being measured.
By aligning our metrics with our goals, we are focusing people’s attention on the things that are important to us.
Provide feedback: Providing regular feedback to the team about the data they help collect has several benefits:
• It helps maintain focus on the need to collect the data. When the team sees the data actually
being used, they are more likely to consider data collection important.
• If team members are kept informed about the specifics of how the data is used, they are less
likely to become suspicious about its use.
• By involving team members in data analysis and process improvement efforts, we benefit from
their unique knowledge and experience.
• Feedback on data collection problems and data integrity issues helps educate team members
responsible for data collection. The benefit can be more accurate, consistent, and timely data.
Obtain “buy-in”: To have ‘buy-in” to both the goals and the metrics in a measurement program, team members
need to have a feeling of ownership. Participating in the definition of the metrics will enhance this feeling of
ownership. In addition, the people who work with a process on a daily basis will have intimate knowledge of that
process. This gives them a valuable perspective on how the process can best be measured to ensure accuracy
and validity and how to best interpret the measured result to maximize usefulness.
Conclusion
A metrics program that is based on the goals of an organization will help communicate, measure progress towards,
and eventually attain those goals. People will work to accomplish what they believe to be important. Well-designed
metrics with documented objectives can help an organization obtain the information it needs to continue to improve
its software products, processes, and services while maintaining a focus on what is important. A practical,
systematic, start-to-finish method of selecting, designing, and implementing software metrics is a valuable aid.
References
[Basili-88] V. R. Basili, H. D. Rombach., 1988, The TAME Project: Towards Improvement-Oriented
Software Environments. In IEEE Transactions in Software Engineering 14(6) (November).
[Fenton-91] Norman E. Fenton, 1991, Software Metrics, A Rigorous Approach, Chapman & Hall, London.
[Goodman-93] Paul Goodman, 1993, Practical Implementation of Software Metrics, McGraw Hill, London.
 Page 22 The STANDARD Spring 2001/Winter 2000

[Grady-92] Robert B. Grady, 1992, Practical Software Metrics for Project Management and Process
Improvement, Prentice-Hall, Englewood Cliffs.
[IEEE-83] IEEE Standard Glossary of Software Engineering Terminology, ANSI/IEEE Std 729-1983, The
Institute of Electrical and Electronics Engineers, New York, NY, 1983.
[Jones-86] Capers Jones, 1986, Programming Productivity, McGraw Hill, New York.
[Humphrey-89] Watts Humphrey, 1989, Managing the Software Process, Addison-Wesley, Reading.
[Schulmeyer-98] G. Gordon Schulmeyer, James I. McManus, Handbook of Software Quality Assurance, 3rd
Edition, Prentice Hall PTR, Upper Saddle River, NJ, 1998.

Trusting Measurement Results

In
The Chemical and Process Industries
David Files, CIC
The Chemical Interest Committee (CIC) of the Chemi-
cal and Process Industries Division has just sent this
book to Quality Press for publication prior to May 2001.
At the 2001 AQC the CIC will highlight key points from
the book in session T304 on Tuesday, May 8, at 3:45
PM.
All members of the CIC, whether they generated or
used measurement results, agreed that LACK OF
TRUST was the BIG issue with Measurement Results.
The work and costs being generated by lack of trust are
a total waste! Thus, the working title was “The Mea-
surement System Says: ‘Trust this number’; You ask:
‘Why Should I?’.” If we could write a good comeback to
this we would have a book that had a chance of helping
build trust in measurement results and making our lives
easier.
In Part 1 we introduce our reasons for writing the book
via stories and identify the two audiences for whom we
have tried to write this book.
1. People who are asked to make business decisions
based on the numbers they are given.
2. People who are asked to generate the numbers
given to the people making business decisions.
Both audiences need to quantify their trust of the
numbers and know how to use the numbers based on
the level of trust they have in them.
To help each of us quantify our trust in the numbers
Part 2 of the book examines every step in the measure-
ment system and what it means to our trust in the
numbers. A checklist is provided as a tool for measur-
ing our trust in each measurement step and arriving at
an overall measure of our trust in the reported number.
As with any physical product, design of the measure-
ment system is critical to its capability to report trust-
worthy numbers. Design of a measurement system
begins with the definition of the choice to be made
based on the measurement result that will be reported.
Only through knowing the choice to be made can a
measurement system be designed that has the capa-
bility of reporting a trustworthy number.
The actual measurement process starts with taking the
sample and ends with reporting the measurement
result. Nothing is more important to a trustworthy result
than the taking of the sample from the material. No
measurement system can compensate for a bad
sample. Also, proper sampling is essential for testing
cost management.
Part 3 discusses what users inside and outside of the
measurement system can do with measurement val-
ues and measurement results. All users must be in-
formed of the nature of the number. Is it a measurement
value or a measurement result? A measurement value
is the output of a single measurement event. Some
refer to measurement value as raw data. A measure-
ment result is calculated based on one or more mea-
surement values. A measurement result can be a
number adjusted for atmospheric conditions, bias in
measurement process, an average of several mea-
surement values, etc., depending on what the mea-
surement system design process determined to be the
best number to report for making the intended busi-
ness decision.
Part 4 looks at the future of measurements in industry.
What are the benefits and risks of the changes that are
being made? What needs to be in place for these
changes to deliver maximum benefit?
We hope the book will be a useful tool that will lead to
better business decisions through “Trusting Measure-
ment Results.”
David Files, Chair CIC
 Spring 2001/Winter 2000 The STANDARD
Profile on: NCSL International
NCSL International was formed in 1961 (as NCSL) to
promote cooperative efforts for solving the common
problems faced by measurement laboratories. Today,
NCSL International has over 1500 member organiza-
tions from academic, scientific, industrial, commercial
and government facilities around the world. This wide
representation of experience provides members a rich
opportunity to exchange ideas, techniques, and inno-
vations with others engaged in measurement science.
NCSL International is a nonprofit organization whose
membership is open to any organization with an inter-
est in the science of measurement and its application
in research, development, education, or commerce.
VISION
Promote competitiveness of NCSL International Mem-
ber Organizations by improving the quality of products
and services through excellence in calibration and
testing.
MISSION STATEMENT
The mission of NCSL International is to advance tech-
nical and managerial excellence in the fields of Metrol-
ogy, Measurement Standards, Instrument Calibration,
and Test and Measurement, through voluntary activi-
ties aimed at improving product and service quality,
productivity, and the competitiveness of Member Or-
ganizations in the national and international market-
place.
NCSL International accomplishes its mission through
activities whose purposes are to:
• Promote voluntary and cooperative efforts to
solve common problems faced by its Mem-
ber Organizations.
• Promote and disseminate relevant informa-
tion that is important to its member organi-
zations.
• Formulate consensus positions of the mem-
bership when requested by outside organi-
zations and government bodies that will serve
all or segments of the Member Organizations.
• Advance the state-of-the-art in Metrology and
related activities in both the technical and
the management area.
• Provide liaison with technical societies, trade
associations, educational institutions, and other
organizations or activities that have common
interests.
• Assess Metrology requirements and develop
uniform, recommended practices related to
the activities of the membership.
• Provide a forum to accomplish the objectives
of NCSL International through conferences,
Page 23
regional and sectional meetings, committee
activities, and publications.
• Serve as an effective channel to assist vari-
ous national laboratories disseminate infor-
mation to Metrological communities, and to
collect and present information to strengthen
and improve national measurement systems
and the horizontal linkages between these
systems
NCLS President’s Message: An Interview with John
Ragsdale , NCSLI President
NCSLI at 40, the Best Years of Our Organi-zational
Lives
As we enter the new century, and NCSL International
celebrates its 40th anniversary, it will be an especially
memorable year as we reflect on past accomplish-
ments and focus on the opportunities that the future
presents. This year also marks the 100th anniversary
of the National Institute of Standards and Technology.
The following interview will recognize and celebrate
both of these significant milestones at the NCLS an-
nual Workshop and Symposium that will be held in
Washington, D.C. July 29 - August 2, 2001. The
Conference Committee is already hard at work to
ensure that the 2001 conference will be the best ever.
Please begin making your plans to attend because this
will truly be a conference to remember and one that you
definitely won’t want to miss.
The Standard: What is the State of the Society?
Ragsdale: At the October Board meeting held in
Orlando, Fla., the Board completed the drafts of the
Long Range Plan for 2001-2005 and the budget for
2001. We expect to approve the final versions of both
plans at the January Board meeting. As part of the
planning process for 2001, the Board established
several key focus areas to ensure NCSL International’s
continuing success. The three principal areas of focus
are: Membership Growth, Financial Management, and
Conference Planning.
Membership Growth: International membership con-
tinues to grow at an accelerating pace, while domestic
membership growth over the past three years has been
relatively flat. However, with the continued expansion
of the global marketplace, more and more companies
whose traditional focus has been on the domestic
economy are being impacted, directly and indirectly, by
international quality initiatives, documentary standards,
quality system registration and laboratory accredita-
tion. These companies can definitely profit from mem-
bership in NCSL International, and they present an
excellent opportunity for significant growth in our mem-
 Page 24 The STANDARD
bership. To take advantage of this opportunity, major
focus will be placed on increasing the membership by
continuing to recruit members from our more traditional
customer base while tapping into segments of the
economy that have not previously been involved in
NCSL International. To support this effort, and ensure
our success, two important committee chair positions
have recently been filled. Larry Yates will chair the
Membership Committee and Jim Tavernier will chair
the Publicity Committee. Working together they will
develop a strategic marketing plan for the organization
designed to enhance the recruitment process and to
identify and access new areas of membership.
Another action of significance from the October Board
meeting related to membership was the formation of an
Ad Hoc Committee to study the current structure of
NCSL International. The Committee will make recom-
mendations for enhancing the alignment of the organi-
zation to improve services provided to the membership
and to more evenly distribute the workload for the
various Vice Presidential areas. The Committee will
report the results of its study to the Board at the January
meeting.
The Standard: What is the Financial Management
process that NCLS uses?
Ragsdale: Managing the financial health of the orga-
nization is an ongoing challenge for the Board and the
Business Office Staff. Both continually look for ways to
reduce operational costs while providing enhanced
and expanded services to the membership. This chal-
lenge increases with each passing year as NCSL
International’s influence and participation continues to
grow in regional metrology organizations, standards
writing bodies and accreditation cooperative organiza-
tions. Leon Barnes, Treasurer, and Craig Gulka, Busi-
ness Manager, have been working closely with our
accounting firm to transition the current-cash based
financial system to an accrual-based system. The new
system will give more accurate, up-to-date information
that will enable us to keep a closer watch on our
expenses and allow us to better manage our finances
and plan for the future success of the organization.
Conference Planning: As mentioned previously, the
Conference Committee is hard at work planning the
2001 Workshop and Symposium. They performed an
outstanding job last year with the Toronto conference
and raised the quality for our conference to a new level
of excellence. While Toronto will be a hard act to follow,
I’ m certain that they are up to the challenge. They are
continuing to strengthen the technical sessions, ex-
pand the pre/post conference tutorial programs, and
are planning a number of recognition events celebrat-
ing the 40th Anniversary of NCSL International and the
100th Anniversary of NIST.
Spring 2001/Winter 2000
The Standard: What Types of challenges does the
New year bring?
Ragsdale: The new year begins with quite a turnover
on the Board of Directors. Bill Sorrells and Jim Patterson
are both retiring, and we wish them a long and enjoy-
able retirement. Gary Shuler is also stepping down
from the Board after 10 years of service. Gary plans to
remain active in NCSL International but at a reduced
level of involvement. We extend a warm welcome to
four new Board members whose terms begin January
1, 2001. Georgia Harris will become Vice President of
Publications. Georgia previously served on the Board
(1994-97) as Vice President of Measurement Science
and Technology. Carol Hockert, previously Region 11
Coordinator, will become Vice President of the Central
Division; and John Wehrmeyer, former chairperson of
the ANSI/NCSL Z540-1 Writing Committee, will be-
come Vice President of Documentary Standards Appli-
cations. Jeff Gust, who was appointed last year to fill
the new position of Vice President Northeast Division,
was elected to the Board and will continue in his
present position. Other changes to the Board include
Charlie Motzko moving to the Executive Vice President
position and Mike Suraci’s re-election to the Board and
assignment as Vice President of Conference Manage-
ment. Dave Nebel will move to Vice President of
Education and Training, Woody Tramel will replace
Charlie Motzko as Vice President of Operations and
Marketing, and Ed Pritchard has been appointed to fill
the vacant Vice President position for the Southeast-
ern Division. Dr. Klaus Jaeger leaves the board after
completing his term of office as Immediate Past Presi-
dent. Although Klaus will be leaving the Board, I hope
that he will continue his involvement with NCSLI.
Other retirements of note are those of Ernest Garner
and Sharrill Dittmann of NIST. Ernest and Sharrill have
a long record of service to NCSL International, and
their presence and wise counsel at our Board meetings
will be greatly missed. We wish them well in their
retirement. On behalf of the entire Board I extend our
thanks and appreciation to Dave Abell for his wisdom
and strong leadership as President during the past
year. During his tenure we had our most successful
conference and for the first time held the conference
outside of the United States. Dave continued to build
strong alliances with the North American National
Measurement Institutes and held meetings with the
Directors and senior management of NIST (US), IMNS/
NRCC (Canada) and CENAM (Mexico). He also repre-
sented NCSL International at several international
conferences and events throughout the year. I look
forward to working with Dave during my term as Presi-
dent and will rely heavily on him for guidance and
support.
 Spring 2001/Winter 2000 The STANDARD Page 25

The Standard: What message do you have for the Papers, Panels and Workshops:
readers of The Standard? The Workshop & Symposium offers papers, panels,
Ragsdale: I encourage each of you to become more and workshops that explore the theme and are orga-
involved in the activities of NCSL International. Volun- nized into three categories:
teers are needed at all levels of the organization but • Theoretical
especially at the Region and Section level. The Region • Applied
and Section meetings are the backbone of our organi- • Management/Quality
zation and our most important point of contact with the
general membership. It is through these meetings that Networking:
we receive the feedback that is crucial in planning and The Workshop & Symposium provides many opportu-
developing new products and services to meet the nities to meet other individuals in the field of Metrology
ever-changing needs of our customers. It is especially to work out problems, gain new perspectives, and
gratifying to see so many of our Regions and Sections solve situations similar to yours.
turn in continued excellent attendance and impressive
Committees:
presentation agendas.
Join with the people who are doing the inside work on
Summary Committees such as:
During the last decade of the 20th century there has • Accreditation Resources
been a significant increase in the recognition of Metrol- • Airline Metrology
ogy and the important role it plays in improving produc- • ANSI/NCSL Standards Writing Committee
tivity, product quality, product defect reduction and, • Automatic Test & Calibration Systems
perhaps of greatest importance, its impact on global • Automotive Committee
trade issues. • Benchmarking Programs
This increased recognition has made a significant shift • Calibration/Certification Procedures
in NCSL International’s traditional membership to a • Chemical & Bio Defense
more widely diverse membership base. Companies • Education Systems
from the Chemical, Automotive, Optoelectronic, Air- • Equipment Management Forum
line, Healthcare, Testing Laboratory, Wireless Com- • Glossary
munication, and Food and Drug industries have recog- • Healthcare Metrology
nized the impact that Metrology has on their competi- • International Measurements Coordination
tiveness. • Intrinsic & Derived Standards
• Laboratory Evaluation Laboratory
In response to this challenge they have formed strong
• Facilities
committees within NCSL International to address mea-
• Measurements Comparison Program
surement issues and requirements critical to their
• Metrology Practices
business success. Additionally, the emergence of new
• Personnel Training Requirement
technologies, such as MEMS (Micro Electro Mechani-
• Small Business Initiative
cal Systems) and other nanotechnology applications,
• Training Resources
web based interactive training and metrology inter-
• Utilities
change will require new standards, measurement pro-
cesses, measurement techniques, and management Exhibits:
tools to support their development and application. Meet face-to-face with key company executives and
As we move into the new century, conformity assess- technical experts from 150 leading manufacturers sup-
ment, which includes metrology, standards, and labo- plying products and services to the measurement
ratory accreditation, will become even more important community. Learn about:
to the economic competitiveness of NCSL • New Equipment Demonstrations
International’s membership. An additional highlight of • Applications Information
the 2001 Conference there will be a recognition of • Problem-Solving and Networking
NCSL International’s 40th anniversary and the 100th
anniversary of the US National Institute of Standards Join other metrologists in Washington, D.C., July
and Technology (NIST). 29 - August 2, 2001

There will be several events celebrating these mile- TO CONTACT NCSL:

stones during the Conference, and there will be tours
of the NIST laboratories at Gaithersburg, Md., on
Friday after the conference.
NCSL International © 1996-2001 All Content.
1800 30th Street, Suite 305B, Boulder, Colorado
80301-1026
Tel: (303) 440-3339 Fax: (303) 440-3384 info@
ncslinternational.org
Page 26 The STANDARD Spring 2001/Winter 2000
Spring 2001/Winter 2000 The STANDARD Page 27
Page 28 The STANDARD Spring 2001/Winter 2000
Spring 2001/Winter 2000 The STANDARD Page 29
Page 30 The STANDARD Spring 2001/Winter 2000
Spring 2001/Winter 2000 The STANDARD Page 31
Page 32 The STANDARD Spring 2001/Winter 2000

Operator WR(B) WR(C) Part

Average
Spring 2001/Winter 2000 The STANDARD Page 33
Page 34 The STANDARD Spring 2001/Winter 2000

Important Upcoming Events

Conference Dates Location Contact Number

AQC 2001 5/7 - 9/2001 Charlotte, NC 1-800-248-1946

NCSL 2001 7/29 - 8/2/2001 Washington, DC 303-440-3339

MQD/NIST Sept. 13-14, 2001 Washington, DC 301-869-4520

 Spring 2001/Winter 2000 The STANDARD
News Update
A2LA One of Three Signatories of
NACLA MRA
On September 29. A2LA became one of the first
three accrediting bodies (ABs) recognized by The
National Cooperation for Laboratory Accreditation
(NACLA), when representatives of the three ABs signed
the NACLA mutual recognition arrangement (MRA).
Pete Unger, President, signed for A2LA. The other two
signatory bodies are NVLAP and ICBO Evaluation
Service.
NACLA is the private-sector, nonprofit voluntary
organization whose mission is to coordinate U.S. labo-
ratory accreditation by granting recognition to compe-
tent ABs.
The basis for NACLA recognition is a three-step
process: Careful review of the AB’s documents and
procedures; a thorough on-site evaluation by a team of
NACLA experts to determine the AB’s compliance with
NACLA procedures and the international standard for
AB’s (ISO/IEC 58); and review of the evaluation team’s
report and recommendation by a NACLA Acceptance
Panel.
NACLA signatory AB’s commit to the following:
• To use equivalent procedures in the accreditation
of laboratories under ISO/IEC Guides 58 and
25 (17025)
• To recognize the test reports and calibration
laboratories accredited by other signatory bodies
as being technically equivalent
• To promote the acceptance by all users of
test reports and calibration certificates issued
by their accredited testing and calibration
laboratories
• To investigate all complaints resulting from
test reports and calibration certificates issued
by their accredited testing and calibration
laboratories
• To inform the other signatories as soon as
possible, through the NACLA Secretariat, of
any significant changes that have occurred
or will occur in the status and/or operational
practices of their accreditation body
A2LA has been an extremely active member of
NACLA since its founding three years ago. Roxanne
Robinson is currently a member of the group’s Board of
Directors, and Pete Unger is Chair of the Training
Committee.
Page 35
New Traceability Policy Issued
The “Calibration Accreditation Policy” and the “waiver
form” process were formally replaced by the “Trace-
ability Policy” on August 28, 2000. The new Traceabil-
ity Policy continues to require that A2LA-accredited
laboratories obtain accredited calibration reports from
their calibration providers or obtain calibrations directly
from a National Metrology Institute (NMI).
During the laboratory’s on-site assessment, the
Assessor will review the laboratory’s calibration records
and determine during the assessment if:
a. the lab can demonstrate that the calibration
was performed by our national metrology
institute, NIST, or by another NMI; or
b. the lab can demonstrate that the calibration
was performed by an A2LA-accredited cali-
bration lab or by a calibration lab that has
been accredited by one of our MRA part-
ners. (Assessors must see a calibration cer-
tificate with the accreditation body logo in-
cluded, or which otherwise makes reference
to its accredited status.)
If the conditions noted above are not met, the
assessor is required to cite a deficiency.
The laboratory should respond to the deficiency
with a corrective action outlining their plans to obtain
accredited calibration services, or else with their justi-
fication for using the current calibration vendors. Labo-
ratories are no longer required to complete the waiver
form. A decision on the acceptability of the corrective
action will be made by knowledgeable staff at A2LA
and the lab will be notified as to whether or not the
response is deemed acceptable.
Since the formal “waiver form” process has been
discontinued, the waiver forms will no longer be pro-
vided. Waiver forms currently in-house or received due
to an assessment conducted before the process was
phased out will be reviewed and processed as usual.
It should be noted that in previous memos to our
assessor corps and to our accredited and applicant
laboratories, we had outlined a plan wherein resolution
of the deficiency would be agreed to by the assessor
and the laboratory. In addition, this earlier plan allowed
the assessor some latitude concerning whether or not
the deficiency should be cited at all. However, some
laboratories and assessors raised the issue of poten-
tial “assessor variability” in deciding whether or not a
given non-accredited calibration provider was accept-
able. In addition, as part of maintaining our status as a
signatory to the APLAC and EA mutual recognition
arrangements, A2LA must monitor the traceability sta-
tus of our accredited laboratories. For these reasons,
the initial plan was revised as noted above.
 Page 36 The STANDARD
If you have any questions about the change of policy
and procedure, any technical staff member should be
able to answer your questions, or you can call Thomas
Adams at 301-644-3219 or contact him via e-mail at
tmadams@a2la.org.
Thank you for your patience during our transition
period!
Electromechanical Advisory Committee
Initiated
A2LA has established a Electromechanical Advi-
sory Committee (EMAC) to provide the association
with guidance in the electromechanical field of testing.
The committee scope will include the development of
accreditation guides for electrical product testing, cov-
ering the following technical areas: electrical, electro-
magnetic compatibility (EMC), product safety, tele-
communications, environmental simulation and elec-
tromechanical (functional). In selecting projects, the
EMAC will coordinate with other A2LA committees and
with other organizations and individuals active in the
electromechanical field, so-as-to minimize duplication
of efforts.
Michael Tedaldi has been appointed Chairman of
the new committee. He holds a Master of Science
Degree in Physics and has extensive experience as an
EMC Engineer, Senior Quality Assurance Engineer,
and Safety Engineer.
Participatory membership on the EMAC is open to
all interested parties. For more information about EMAC
membership, please contact Trace McInturff, A2LA
Program Manager at tmcinturff@a2la.org.
ISO/IEC 17025 Transition Update
The A2LA leadership has adopted a new policy
directive and has published several new documents to
further the organization’s transition from the current
international accreditation standard (ISO/IEC Guide
25) to the new one (ISO/IEC Standard 17025).
Policy Directive: As of January 1, 2001, A2LA will
discontinue full on-site assessments against ISO/IEC
Guide 25 only. Laboratories will have two options;
either a full ISO/IEC 17025 assessment, or an ISO/IEC
Guide 25 assessment with an ISO/IEC 17025 gap
analysis. (Details of this new policy are included in the
Transition Plan chart printed in this newsletter.)
New Documents: In September 2001, all accredited
and applicant laboratories were provided with four new
documents:
• General Requirements for the Accredita-
tion of Laboratories
Spring 2001/Winter 2000
This “green booklet” contains the interpre-
tive guidance for ISO/IEC 17025 only. The
actual requirements of ISO/IEC 17025 are
not contained in the document. Laboratories
are now required to obtain their own copy of
ISO/IEC 17025. A2LA’s web site contains
information on how to order the standard.
• A2LA Policy on Measurement Traceability
The “A2LA Policy on Measurement Trace-
ability” replaces the Calibration Accreditation
Policy and the Waiver Procedure, which have
been removed from circulation. The policy
contains new requirements that have been
added to the ISO/IEC 17025 Assessor Checklist.
Please refer to the related article in this edition
of A2LA News.
• Proficiency Testing (PT) Requirements for
Accredited Testing and Calibration Labo-
ratories
The PT requirements for all fields of accreditation
are printed in this new document. You are
encouraged to review those sections relevant
to you laboratory’s field(s) of testing/calibra-
tion.
• A2LA Interim Policy on Measurement Un-
certainty for Testing Laboratories
Under Interim Policy on Measurement Un-
certainty for Testing Laboratories may respond
to a deficiency (or gap analysis finding) in
this area by submitting an implementation
plan with its corrective action response. The
laboratory will then have a year to obtain
training, write its procedures and estimate
uncertainty where it is applicable.
All these new documents are posted on A2LA’s web
site, www.a2la.org.
A2LA ISO/IEC 17025 Transition Plan for
2001 (revision date: 12/01/2000)
The International Laboratory Accreditation Coop-
eration (ILAC) recently issued the following revised
transition guidance to laboratory accreditation bodies:
“Accreditation bodies shall require that the laboratories
for which they grant and maintain accreditation comply
with the requirements of ISO/IEC 17025 by December
31, 2002. No extra surveillance activities will be re-
quired to confirm this but all accredited labs must have
been through a surveillance, evaluation or re-evalua-
tion before this date. After this date all accreditation
documents must refer to ISO/IEC 17025.”
The chart noted below identifies the A2LA assess-
ment options available to new and renewal laborato-
 Spring 2001/Winter 2000 The STANDARD Page 37

ries for 2001 to ensure that we meet our goal of

demonstrated compliance to ISO/IEC 17025 by July of
2002, six months in advance of the ILAC deadline.
NEW AND RENEWAL ASSESSMENTS
A2LA ISO/IEC 17025 TRANSITION PLAN
(revision date: 12/01/2000)

Assessment Options Documentation Comments

Date
Beginning OPTION I: ISO/IEC 17025 Checklist
January 1, 2001 ISO/IEC 17025
OPTION II: ISO/IEC Guide 25 and Note 1: Response to gaps
ISO/IEC Guide 25 with an 17025 Gaps Checklist - due at annual review
ISO/IEC 17025 Gap document gap analysis by (renewals) or surveillance
Analysis filling out bolded (17025) (new)
items; Assessors will write
up "gaps" in separate form Note 2: Depending on the
attached to deficiency report number and nature of the
that includes the following gaps identified during the
statements. "In order to be gap analysis, an on-site
considered for accreditation revisit may be required to
to ISO/IEC 17025, objective verify compliance.
evidence that the following
items have been addressed
needs to be provided to
A2LA with the annual
review/surveillance."
Beginning OPTION I ISO/IEC 17025 Checklist
July 1, 2001 ISO/IEC 17025
Surveillance Assessments
Surveillance assessment that occur during 2001 must automatically include the ISO 17025 Gap Analysis at
a minimum if the lab was assessed to ISO/IEC Guide 25 during the initial accreditation process. See Option II
above.

An Idea that Changed the World of Metrology Forever:

And the Man Behind It

Most of us never heard of Claude Shannon, which is a
pity. Because Shannon, who passed away last week-
end at the age of 84, had an idea that changed the
world forever, especially the world of metrology and
measurement.
(news.cnet.com/news/0-1004-200-4968313.html).
The year was 1948 and Shannon was a young Bell
Labs scientist who had reached the startling conclu-
sion that all information could be expressed in binary
format–as a series of 1s and 0s. Although the technol-
ogy of the day wasn’t advanced enough to take advan-
tage of these findings immediately, they eventually
became the basis for measurement of digital technolo-
gies as diverse as modems, magnetic storage, the
Internet, and satellite transmissions. Other impressive
contributions in mathematics and cryptography fol-
lowed.
Ironically, Shannon, a distant relative of Thomas
Edison, was perhaps best remembered within the
scientific community for his wacky inventions, such as
the rocket-powered Frisbee, and his freewheeling an-
tics, such as riding his unicycle through the halls while
juggling balls in the air. Thank you, Claude Elwood
Shannon, for all you’ve given us. This one’s for you!
 Page 38 The STANDARD Spring 2001/Winter 2000

A2LA Training Course Schedule

A2LA Quality in Laboratory Training Course Schedule

Please visit our web site (www.a2la.org) for a description of the courses listed below.
Irvine, CA
Jan. 5 ISO/IEC 17025, The New Standard for Laboratories
Charleston, SC
Feb. 26-27 ISO/IEC 17025, and Accreditation
Feb. 28 ISO/IEC 17025, The New Standard for Laboratories
March 1-2 Calibration Laboratory Practices
March 5-6 Measurement Uncertainty
Columbia, MD
March 12 ISO/IEC 17025, The New Standard for Laboratories
Irvine, CA
April 6 ISO/IEC 17025, The New Standard for Laboratories
New Orleans, LA
April 23-24 ISO/IEC 17025 and Accreditation
April 25 ISO/IEC 17025, The New Standard for Laboratories
April 26-27 Calibration Laboratory Practices
April 30-May 5 Measurement Uncertainty
Irvine, CA
June 8 ISO/IEC 17025, The New Standard for Laboratories

2001 Measurement Quality Conference

September 13 – 14, 2001

Location: National Institute of Standards and Tech-
nology, Gaithersburg, MD
Conference Hotel: Gaithersburg Holiday Inn
The 2001 Measurement Quality Conference will be
held at the National Institute of Standards and Technol-
ogy in recognition of the 100th anniversary of its found-
ing as the National Bureau of Standards. The confer-
ence will open with a special, dual session with NIST
retirees delineating the history of NBS/NIST and NIST
managers giving a look into the next fifteen years for
four metrological discipline areas.
This special presentation will be followed by sessions
on:
• Laboratory accreditation in the context of ISO
17025
• Statistics and uncertainties
• Measures of quality
The Conference Banquet at the Holiday Inn on Thurs-
day evening will provide further opportunities for net-
working and discussion of quality issues.
The Conference will wrap up Friday afternoon with
tours of NIST, including a look at the new Advanced
Measurement Laboratory currently under construction
just south of the main cluster of buildings on the NIST
Gaithersburg campus.
Measurement Quality Division Meeting: On Wednes-
day evening, September 12, there will be a meeting of
the ASQ Measurement Quality Division at the Confer-
ence Hotel, beginning at 7:30 PM. Division members
and prospective members are welcome to attend.
Tutorial: On Wednesday, September 12, there will be
a day-long tutorial on aspects of measurement quality
held in NIST North, in the first floor conference room.
Registration and fees for the tutorial are separate from
those of the Conference. Refreshment breaks will be
included. Lunch will be no-host in one of the many
restaurants in the vicinity.
For more information contact Norm Belecki at
n.belecki@ieee.org.
 Spring 2001/Winter 2000 The STANDARD Page 39

Metrology/Technology Books

Metrology
“An Introduction to Measurement and Calibration”, Campbell, Paul
Industrial Press, May 1995
Paperback, $24.95 isbn: 0831130601

“Fundamentals of Temperature, Pressure and Flow Measurements”, 3rd Edition, Benedict,

Robert P.
Wiley, Jul 1984
Hardcover, $131.25 isbn: 0471893838

“Nonlinear Optics”, 4th Edition, Bloembergen, Nicolaas

World Scientific, May 1996
Paperback, $18.90 isbn: 9810225997
Hardcover, $33.60 isbn: 9810225989

“Coherent Optics; Fundamentals and Applications”, Lauterborn, W., Kurz, T. and

Wiesenfeldt, W.
Mar 1995

Paperback, $49.00 isbn: 0387583726

“Eye and Visual Optical Instruments”, Smith, George

Cambridge University Press, Jun 1996

Paperback, $57.70 isbn: 0521478200

“Geometrical Optics and Optical Design”, Mouroulis, Pantazis

(Oxford Series on Optical and Imaging Sciences)

Oxford University Press, Nov 1996

Hardcover, $80.40 isbn: 0195089316

“Time Domain Optics of Ultrashort Waveforms”, Shvartsburg, A.B.

(Oxford Series on Optical and Imaging Sciences)

Oxford University Press, Aug 1996

Hardcover, $99.00 isbn: 0198565097

“Optical Holography; Principles, Techniques and Applications” 2nd Edition, Harihan, P.

(Cambridge Studies in Modern Optics)

Cambridge University Press, May 1996

Paperback, $41.95 isbn: 0521439655

“Opticks”, Newton, Isaac

Jan 1952

Paperback, $11.65 isbn: 0521439655

“Nonlinear Optics and Quantum Electronics”, Scubert, Max and Wilhelmi, Bernd

(Wiley Series in Pure and Applied Optics)

Wiley, Jul 1986

Hardcover, $172.20 isbn: 0471088072

“Nonlinear Optics”, Moloney, Jerome V. and Newell, Alan C.

(Advanced Topics in the Interdisciplinary Mathematical Sciences)

Addison-Wesley, Apr 1992

Hardcover, $53.35 isbn: 0201510146

 Page 40 The STANDARD Spring 2001/Winter 2000

“The Physics of Moire Metrology”

(Wiley Series of Pure and Applied Optics) by Kafri, Oded
Wiley, 1990
Hardcover, $88.15 isbn: 0471509671

“The Principles of Nonlinear Optics”

(Wiley Series of Pure and Applied Optics) by Shen, Y.R.
J. Wiley, May 1984
Hardcover, $121.80 isbn: 0471889989

“Absorption and Scattering of Light by Small Particles” by Bohren, Craig F.

Mar 1983

Hardcover, $156.45 isbn: 047105772X

Physical/Dimensional Measurements
“Applied Measurement Engineering; How to Design Effective Mechanical Measurement
Systems”
Wright, Charles P.
Printice Hall, Aug 1994
Hardcover, $70.35 isbn: 0132534770

Statistics
“A Handbook of Introductory Statistical Methods”, Cox, C. Philip
Wiley, Jan 1987
Hardcover, $97.60 isbn: 0471819719

“Statistics for Experimenters; An Introduction to Design, Data Analysis and Model Building”
by Box, George E.P.

Wiley, Jun 1978

Hardcover, $73.45 isbn: 0471093157

“Design and Analysis of Reliability Studies; The Statistical Evaluation of Measurement Errors”
Dunn, Graham
Jan 1993
Hardcover, $94.45 isbn: 0470220651

Temperature
“Temperature Measurement” by Liptak, Bela G.

Chilton Book Co., Jun 1993

Hardcover, $32.95 isbn: 0801983851

“Principles and Methods of Temperature Measurement”, by McGee, Thomas D.

Wiley, May 1988

Hardcover, $89.20 isbn: 0471627674

“Thermodynamic Properties of Cryogenic Fluids”, by Jacobsen, R.T.

Hardcover, $99.75 isbn: 0306455226

“Thermodynamics and an Introduction to Thermostatistics” 2nd Edition, Collen

Wiley, Aug 1985

Hardcover, $69.53 isbn: 0471862568

“Advanced Cryogenics”, by Bailey, Colin Alfred

(The International Cryogenics Monograph Series)
Paperback, $68.25 isbn: 0306304589
 Spring 2001/Winter 2000 The STANDARD Page 41

“Experimental Techniques in Low-Temperature Physics”, 3rd Edition, White, Guy K.

(Monographs on the Physics and Chemistry)

Clarendon Press, Aug 1987

Paperback, $51.75 isbn: 019851381X

Acoustics
“Principles of Vibration and Sound”, Rossing, Thomas and Fletcher, Nelville H.

Springer-Verlag,

Hardcover(Oct 1994) $54.95 isbn: 0387943048

Paperback (Dec 1994) $34.50 isbn: 0387943366

“The Physics of Sound” 2nd Edition, Berg, Richard E.

Prentice Hall, Oct 1994

Hardcover $74.55 isbn: 0131830473

“The Science of Sound” 2nd Edition, Rossing, Thomas

Addison-Wesley Publishing Co., Jan 1990

Hardcover, $60.30 isbn: 0201157276

“Theoretical Acoustics”, Morse, Philip M

Princeton University Press, Dec 1987

Paperback, $51.98 isbn: 0691024014

“Ultrasonic Absorption; An Introduction to the Theory of Sound Absorption

and Dispersion in Gases, Li”, Bhatia, A.B.
Dover Publications, Sep 1985
Paperback, $10.76 isbn: 0486649172

“Fundamentals of Acoustics”, 3rd Edition, Kinsler, Lawrence E., Coppens, Alan B., Sanders
Jan 1982
Hardcover, $105.53 isbn: 0471029335

“Nonlinear Acoustics”, Naugolnykh, K.A. and Ostrovsky, L.A.

1994
Hardcover, $85.00 isbn: 1563963388

“Handbook of Human Vibration”, Griffin, Michael J.

Academic Press, Dec 1996

Paperback, $62.95 isbn: 0123030412

“Fundamentals of Piezoelectricity” by Ikeda, T.

(Oxford Science Publications)

Oxford University Press, Jul 1990

Hardcover, $117.60 isbn: 0198563396

“AIP Handbook of Condenser Microphones; Theory, Calibration and Measurements”

Wang, George, S.K.
American Institute of Physics, 1995
Hardcover, $85.00 isbn: 1563962845

Communications
“Communication Network Test and Measurement Handbook”
Coumbs, Clyde F.
McGraw Hill, Sep 1997
Hardcover, $89.50 isbn: 0070126178
 Page 42 The STANDARD Spring 2001/Winter 2000

General Science — (Best Sellers)

“Fermat’s Last Theroem:

Unlocking the Secret of an Ancient Mathematical Problem”

by Amir D. Aczel
Four Walls Eight Windows, Oct 1996
Hardcover, $16.20 isbn: 1568580770

“Visions: How Science Will Revolutionize the 21st Century”

by Michio Kaku
Anchor Books, Oct 1997
Hardcover, $22.45 isbn: 0385484984

“Emotional Intelligence”, by Daniel Goleman

Bantam Books, Oct 1995
Hardcover, $21.56 isbn: 055309503X

“The Fabric of Reality”, by David Deutsch

Penguin, Aug 1997
Hardcover, $24.25 isbn: 0713990619

“The Large, the Small and the Human Mind”, by Roger Penrose, Malcolm Longair and
Stephen Hawking
Cambridge University Press, Apr 1997
Hardcover, $17.95 isbn: 0521563305

“Eistein’s Mirror”, by Tony Hey and Patric Walters

Cambridge University Press, Aug 1997
Hardcover, $76.94 isbn: 0521435048

“Pythegoras’ Trousers: God, Physics and the Gender Wars”, by Margaret Wertheim
Times Books/Random House, Aug 1995
Hardcover, $20.70 isbn: 081292200X

“Chaos Theory Tamed”, by Garett P. Williams

National Academy Press, Sep 1997
Hardcover, $31.45 isbn: 0309063515

“Why Aren’t Black Holes Black?; The Unanswered Questions at the Frontiers of Space”
by Robert M. Hazen and Maxine Singer
Anchor Books, May 1997
Paperback, $11.65 isbn: 0385480148
 Spring 2001/Winter 2000 The STANDARD Page 43

Measurement Uncertainty
Reference and Guidance Documents
NIST www.physics.nist.gov/pubs/guidelines

NIST Technical Note 1297, 1994 Free

“Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results”
NIST Special Publication 805, 1990 Free
“Uncertainty and Accuracy in Physical Measurements”

ISO

ISO, 1995 cost $

“Guide to the Expression of Uncertainty in Measurement”
NCSL www.ncslinternational.org/publications/pubslist.cfm 303-440-3339
ANSI/NCSL Z540-2, 1997 cost $
“US Guide to the Expression of Uncertainty in Measurement”

NCSL RP-12, 1995 cost $

“Determining and Reporting Measurement Uncertainties”
ASME www.asme.org 1-800-843-2763
ANSI/ASME PTC 19.1, 1998 cost $
“Measurement Uncertainty”

UKAS www.ukas.com/pdfs/M4/PDF 44 181 917 8400

UKAS M3003, 1997 cost $
“The Expression of Uncertainty and Confidence in Measurement”
UKAS LAB 12, 2000 Free
“The Expression of Uncertainty in Testing”

EA www.european-accreditation.org
EA-4/02-S1, 1997 Free
“Expression of Uncertainty of Measurement in Calibration”
SINGLAS www.sac_sci.org.sg 65 334 8566

SINGLAS Technical Guide 1 cost ?

EURACHEM www.vtt.fi/ket/eurachem/quam2000-pl.PDF 353 1 802 5800
EURACHEM/CITAC Guide, 2000 Free
“Quantifying Uncertainty in Analytical Measurement”

NPL www.npl.co.uk
01937 546000
“A Beginner’s Guide to Uncertainty in Measurement”, 1999, Bell cost ?
AMTMA www.amtma.com 313-642-3312

“Searching for Zero, A Guide for Calibration and Uncertainty Factors in Dimensional
Metrology”, 1991 cost $
 Page 44 The STANDARD Spring 2001/Winter 2000

Measurement Uncertainty

Training and Consulting Sources

American Association for Laboratory Accreditation Frederick, Maryland

301-644-3235 www.a2la.org

Benchmark Technologies Toledo, Ohio

419-843-6691 george.macritchie@benchmark-usa.com
Blue Mountain Quality Resources State College, Pennsylvania
1-800-982-2388 www.coolblue.com
Coast Quality Metrology Systems San Clemente, California
949-492-6321

Community College of Aurora Denver, Colorado

303-365-8425 www.cs.cca.cccoes.edu/metrology
E=MC3 Solutions Wadsworth, Ohio
330-328-4400 emc3solu@aol.com
Excel Partnership Sandyhook, Connecticut
1-800-374-3818 www.excelpartnership.com

Flow Dynamics Scottsdale, Arizona

480-948-3789 www.flow-dynamics.com
HN Metrology Consulting Indianapolis, Indiana
317-849-9577 hsnielsen@hn-metrology.com
Integrated Sciences Group Bakersfield, California
661-872-1683 isgmax@lightspeed.net

Philip Stein Consultants Pennington, New jersey

609-737-9144 pstein@measurement.com
Quametec Detroit, Michigan
810-225-8588 www.quametec.com
Tustin Technical Institute Santa Barbara, California
805-682-7171 training@ttiedu.com

University of Wisconsin-Milwaukee Milwaukee, Wisconsin

414-227-3139
Wavetek San Diego, California
1-800-874-4835 www.wavetek.com
Workplace Training Wayzata, Minnesota
1-800-472-2564 www.state.net/wptraining
Spring 2001/Winter 2000 The STANDARD Page 45

Monthly Measurement Quality Division Regular Membership Totals

6000

5000
Number of Members

4000

3000

2000

1000

0
Apr- May- Jun- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun- Aug- Sep- Oct- Nov- Dec- Jan-
Jul-99 Jul-00
99 99 99 99 99 99 99 99 00 00 00 00 00 00 00 00 00 00 00 01
Members 5168 5161 5149 5221 3774 3949 4095 4212 4285 4412 4487 4517 4604 4606 4623 4671 3490 3635 3751 3836 3886 3994

Months

MQD SEP OCT NOV DEC JAN

REGULAR 4711 4743 4769 4824 4842
NEW 66 66 61 28 88
UNPAID (SUBTRACT) 1092 1020 986 954 920
QUIT DIVISION 50 38 8 12 16
TOTAL 3635 3751 3836 3886 3994
RENEW 81 51 24 24 23
COMP 22 22 24 24 24
SENIOR 183 184 181 182 185
NEW 0 0 0 0 1
UNPAID (SUBTRACT) 5 5 5 5 4
QUIT DIVISION 0 0 0 0 0
TOTAL 178 179 176 177 182
RENEW 2 0 0 0 1
COMP 0 0 0 0 0
FELLOW 35 35 38 38 38
NEW 0 0 0 0 0
UNPAID (SUBTRACT) 1 1 1 1 1
QUIT DIVISION 0 0 0 0 0
TOTAL 34 34 37 37 37
RENEW 0 0 0 0 0
COMP 0 0 0 0 0
HONORARY 2 2 2 2 2
NEW 0 0 0 0 0
UNPAID (SUBTRACT) 0 0 0 0 0
QUIT DIVISION 0 0 0 0 0
TOTAL 2 2 2 2 2
RENEW 0 0 0 0 0
COMP 0 0 0 0 0
STUDENT 49 49 49 49 50
NEW 1 1 0 0 2
UNPAID (SUBTRACT) 15 15 13 13 11
QUIT DIVISION 1 1 0 0 2
TOTAL 34 34 36 36 39
RENEW 1 0 2 0 1
COMP 0 0 0 0 0
SUSTAINING 675 674 673 664 657
NEW 13 10 14 9 20
UNPAID (SUBTRACT) 21 34 39 29 29
QUIT DIVISION 0 0 1 1 4
TOTAL 667 650 647 643 644
RENEW 32 28 35 32 41
COMP 0 0 0 0 0

TOTAL-QUIT DIVISION 51 39 9 13 22
TOTAL-QUIT SOCIETY 1 3 5 1 4
TOTAL DECEASED 0 1 0 0 0
4781
GRAND TOTAL 4550 4650 4734 4780 4898
 Measurement Quality Division Officers
Chair
Duane Allen .................. U.S. Navy, P.O. Box 5000, Code MS11, Corona, CA 92878-5000 909-273-4783 V
e-mail: allendk@corona.navy.mil 909-273-4599 F
Past Chairman
Mark Schoenlein ........... Owens Illinois Plastics Group, One SeaGate 29L-PP, Toledo, OH 43666 419-247-7285 V
e-mail: mark.schoenlein@owens-ill.com 419-247-8770 F
Chair Elect
Open
Vice Chair for Regions
Samuel Windsor ............ Filltronic Comtek 410-341-7751 V
31901 Comtek Lane, Salisbury, MD 21804 410-341-0330 F
e-mail: swindsor@fc-us.com
Treasurer
Colleen Gadbois ........... 535 N.W. 112th Ave., Portland, OR 97229-6116 503-646-1380 V
e-mail: cdblflat@home.com 503-646-1380 F
Secretary
Joseph Filipowicz .......... Alliance Compressors
715 Oakland Dr., Natchitoches, LA 71457 318-356-4570 V
e-mail: joeflip@earthlink.net 318-356-4570 F
Auditing
Karl F. Speitel ................ 14 Kalleston Drive, Pittsford, NY 14534 716-385-838 H
Certification
Christopher L. Grachanen ..... Manager, Standards Engineering
P.O. Box 692000 MS070110, Houston, TX 77269-2000 281-518-8486 V
Compaq Computer Corporation, Corporate Metrology 281-518-7275 F
e-mail: chris.grachanen@compaq.com
Programs
Keith Conerly ................ Dow Chemical Co. 517-638-7058 V
e-mail: Kconerly@dow.com 517-638-6928 F
Membership
Kathy Hoath .................. Superior Technical Training 616-957-7750 V
2419 Brook Dr. S.E., Grand Rapids, MI 49512 616-318-9001 C
e-mail: Khoathk@aol.com 616-957-7751 F
Education
Thomas A. Pearson ...... Automated Technology Associates 317-271-9545 ext. 224 V
1635 Expo Lane, Indianapolis, IN 46214 317-271-7974 F
e-mail: tap@ataindy.com
Publications
J.L. Madrigal .................. Oxford Worldwide Group 801-374-1790 V
1045 South Orem Blvd., Orem, UT 84058 801-374-1790 F
e-mail: jim_oxford@msn.com
Newsletter Editor
Frank Voehl ................... St. Lucie Press, 280 Lake Drive, Coconut Creek, FL 33066 954-972-3012 V
e-mail: FVoehl@aol.com 954-978-0643 F
Historian
S.D. (Sal) Scicchitani .... 203 Golf Club Drive, Langhorne, PA 19047

Standards Committee Representative

Dan J. Harper ................ 535 N.W. 112th Ave., Portland, OR 97229-6116 503-646-1380 H
e-mail: dharper.hqa@home.com 503-646-1380 F
NCSL Representative
Christopher L. Grachanen ..... See Certification
Simmons Scholarship
Norm Belecki ................. 7413 Mill Run Dr., Deerwood, MD 20855-1156 301-869-4520 H
e-mail: n.belecki@ieee.org
Website Manager
George A. MacRitchie ... Benchmark Technologies 419-843-6691 V
e-mail: george.macritchie@benchmark-usa.com 419-843-7218 F

Please notify the editor of any errors or changes so that this list can be updated.
REGIONAL COUNCILORS
Region 1
Joseph Califano, Hemagen Diagnos-
tics, Inc., 40 Bear Hill Road, Waltham,
MA 02154 • (417) 890-3766, FAX (617)
890-3748
Region 2
Karl F. Speitel, 14 Kalleston Drive,
Pittsford, NY 14534 • (716) 385-1838
Region 3
Eduardo M. Heidelberg, Carter
Wallace, 61 Kendall Dr., Parlin, NJ
08859 • (609) 655-6521, FAX (609)
655-6736
Region 4
Alex Lau, Imperial Oil, 111 St. Clair
Ave W, Toronto, Ont, Canada M5W-
1K3 • (416) 968-4654, FAX (416) 968-
5560, E-mail: alex.lau@esso.com
Region 5
Open
Region 6
J.L. Madrigal, Brigham Young Univ.,
Dept of Statistics, BYU, 222 TMCB,
Provo, UT 84602 • (801) 378-7357,
FAX (801) 378-5722, E-mail: madrigal
@byu.edu
Region 7
Rolf B.F. Schumacher, Coast Quality
Metrology Systems, Inc., 35 Vista Del
Ponto, San Clemente, CA 92672-3122
• (949) 492-6321, FAX (949) 492-6321
REGIONAL
Region 8
Frank Weingard, Actco Metrology Ser-
vices, 202, Westview Dr. Meadville, PA
16335 • (800) 382-0393, FAX (814)
337-8288, E-mail:
actomet@toolcity.net
Region 9
Dr. Henrik S. Nielson, HN Metrology
Consulting, Inc., 5230 Nob Lane, India-
napolis, IN 46226 • (317) 849-9577
voice and fax, E-mail: hsnielson
@worldnet.att.net
Region 10
George A. MacRitchie, CQE, PE,
Benchmark Technologies Corp., 3161
N. Republic Blvd., Toledo, OH 43615-
1507 • (419) 843-6691, FAX (419) 843-
7218, E-mail: george.macritchie
@benchmark-usa.com
Region 11
Raymond Perham, Michelin Tire Corp.,
Rt 4 Antioch Church, P.O. Box 2846,
Greenville, SC 29605 • (864) 458-1425,
FAX (864) 458-1807, E-mail:
ray.perham@us.michelin.com, or
home E-mail: r.perham007@aol.com
Region 12
Donald Ermer, University of Wiscon-
sin Madison, 240 Mechanical Engineer-
ing Bldg., 1513 University Avenue,
Madison, WI 53706-1572 • (608) 262-
2557
Region 13
Thomas A. Myers, Bellevue Univer-
sity, PMP, CQM, 1000 Galvin Rd. S.,
Bellevue, NE 68123 • 1-800-756-7920
ext. 3714, FAX (402) 293-2035, E-
mail: tmyers@scholars.bell
Region 14
Chuck Carter, C.L. Carter, Jr. & Asso-
ciates, Inc. 1211 Glen Cove Drive,
Richardson, TX 75080 • (972) 234-
3296, FAX (972) 234-3296, E-mail:
asqccarter@aol.com
Region 15
Bryan Miller, Champion International,
Inc., P.O. Box 189, Courtland, AL 35816
• (205) 637-6735, FAX (205) 637-5202
Region 25
Nouman Ali Khan, Descon Engineer-
ing Ltd, 38 Sir Agha Khan III Rd., Lahore
54000, Pakistan • V 92-42-6365134,
FAX 92-42-6364049, E-mail: Nakhan
@descon.com.pk
Please notify the editor of any
errors or changes so that this list
can be updated.
MAP
 The Journal of the Measurement Quality Division
American Society for Quality

The Standard
Non-Profit
Organization
U.S. Postage
PAID
Milwaukee, WI
Permit No. 5419
American Society for Quality
Measurement Quality Division

Spring 2001
Winter 2000

American Society for Quality

Measurement Quality Division
611 E. Wisconsin Ave
P.O. Box 3005
Milwaukee, WI 53201-3005
TIME VALUED MATERIAL