Professional Documents
Culture Documents
34/9, 1805-1811
(1988)
for Estimating
Carryover
Thomas W. Stephens
Accurate
and precise algorithms
for estimating
within-method carryover,
based on the minimization
of a unique carryover sum of squares,
and between-method
carryover,
based on a weighted
Deming
regression
of first sample
recovery
vs carryover-corrected
true recovery,
are described and compared
with traditional
methods by use of a
Additional
error
Keyphrases:
Monte
random-access
analysis
Carlo simulation
weighted
Deming
analytical
regression
Interaction
between
assays in automated
or manual analytical
systems,
operating
in either batch or random-access
modes, has been a traditional
problem
in clinical
chemistry.
In batch analyses,
this interaction
leads to inaccurate
estimation
of the concentration
of an analyte
in a sample
that follows
a highly
atypical
sample
in a run sequence
(within-method
carryover).
Alternatively,
in random-access
analyses,
reagents
that
interfere
with
other
analytical
methods
alter the accuracy
of analyte
determinations
in
samples that are the first to follow a transition
between the
methods
(between-method
carryover).
The carryover
factor is a variable
that was developed to
estimate
the degree of interaction
in a batch analytical
environment
(1). This traditional
algorithm
has been utilized with little change since its inception.
Dixon (2) derived
the carryover
factor more rigorously
and thereby
altered
slightly the equation used to estimate
its magnitude.
New
developments
in instrumentation
and methodology
have
reduced
the carryover
problem in some situations
(3) and
increased it or not changed it in others (4). Until now, the
performance
of the traditional
algorithm
and the influence
of such experimental
factors as analytical
precision, number
of replicates, and concentrations
of standards
was not determined.
currently,
there is no quantitative
method for
carryover between-methods
(5). The introduction
of random-access
analyzers
created
a new problem
of
analytical
interaction
that did not previously
exist.
The fact
that random-access
analyzers
have become commonplace
in
the clinical
laboratory
has increased
the need for a method
to estimate
the magnitude
of this type of error.
Carryover
estimation
is a necessary
step in the complete
evaluation
of an analytical
methodology
in the clinical
chemistry
laboratory
of today. Statistical
evaluations
of
carryover-estimation
algorithms
and of the factors
that
influence
this estimation
are necessary
to understand
the
accuracy
and precision
of these processes
under the various
analytical
environments
expected.
In the absence of such
statistical
information,
the design of an experiment
to
estimate
carryover
is left to the intuitive
approach.
Here I describe
a new approach
to estimating
withinmethod carryover,
based
on the minimization
of a unique
carryover
sum of squares.
I have evaluated
the new and
traditional
algorithms
for precision
and accuracy
over a
range of analytical
variables
(analytical
precision, number
of replicates per group, number of samples per run, ratio of
sample
means)
by a Monte
Carlo study,
i.e., by using
pseudo-random
numbers
generated
to simulate
analytical
requirements
for a large set of data samples and then
assessing
the accuracy
and precision
of the estimates
so
obtained.
As I will show, the new algorithm
is both more
nearly accurate under almost
all conditions
and more precise. I also present an algorithm
for estimating
betweenmethod carryover,
based on weighted
Deming regression of
recovered
first-sample
results vs carryover-corrected
secondsample results
for each set of samples that follow a methodto-method
transition;
the accuracy
and precision
of this
algorithm
are evaluated
over a range of analytical
variables.
estimating
MaterIals
and Methods
Programs
were written
in the C language,
compiled
with the Microsoft
C compiler and run on a Tandy 1200
computer
equipped
with an 8087 floating-point
coprocessor.
The source code for within-method
carryover
estimation
by
the minimization
of carryover
sum of squares
(MCSS)
method is written
to assume
a minimum
of library
support
for maximum
portability.
For ease of comparison,
the data structure
in a run was
set up to be compatible
with the traditional
method
of
carryover
estimation.
These data consist
of a series of
responses
for r replicates
per group, v concentrations
of
sample (v
2, or high and low for this study), and a sets of
groups
per run. Limitation
to this data structure
is not
necessary
for the MCSS method of carryover
estimation,
however,
and the investigator
has greater
flexibility
in
experimental
design with the new method.
=
The traditional
algorithm
is not applicable
to betweenmethod carryover,
found in random-access
analysis, because
this situation does not fit the basic model for within-method
Department
of Biochemistry,
Lilly Research Laboratories,
Lilly and Co., Lilly Corporate
Center, Indianapolis,
IN 46285.
Received January 26, 1988; accepted May 12, 1988.
carryover;
Eli
Available from the author by request
on 5#{188}-inch
diskettes
in PC DOS format.
CLINICAL CHEMISTRY,
in compiled
or source form
1805
The simplest
linear
is:
carryover
Y1=(1-k)X1+kY1_,+ej
where:
Y1
X
k
variation
Sequence
(S11,,521,,
S121,S221,
S112,5212,
S1,S222,
.,
. .
.,
.,
.,
S,,11,
Srj,
(2)
and Y
The
response
carryover
(Y1, next
kjm
1 or 2, m
for sample
factor
j,m
Yr,
ito
S.
is traditionally
s,j
next
<2
determined
-
Yr_i,
j,m)1(r,j,m
when
as:
next
j,m)
(3)
where
next j,m refers
to the next set in the sequence,
whether this refers to an increase
in the j or m index. This
unusual notation becomes necessary
because
of the order
dependence for this sequence. Usually indices are ordered by
the frequency
and sequence of variation;
however,
in this
case, the second index must be the concentration
since it is
varied
second after replicates
even though concentration
is
varied
less than sets (in this case, where v < s). k, determined for each transition
between-groups
(concentrations),
is averaged
as an estimate
for the entire run.
(>>.kjm)/(2
1)
j<2
when
(4)
ss
Y)2]
Y)2]
(6)
CLINICAL CHEMISTRY,
in a lack of equivalence
between positions in the sequence
(samples immediately
following
a group of another concentration
of sample are expected
to be either
high or low,
depending
on the difference
between
samples).
Analytical
also
influences
this
statistic
and,
when
the
data
are corrected
for carryover,
the mean square
of the carryover sum of squares is equivalent
to the analytical
variance.
The MCSS algorithm
works by making
successive quadratic estimates
of the carryover
factor that minimizes
SS,
The best estimate
of the carryover
factor is that which
minimizes
SS. Either SS,, or 55w can be used as an indicathi
of the closeness to the least-squares
carryover
factor,
but SS,
gives less biased estimates.
The algorithm
proceeds by
making a series of guesses at the carryover
factor, and SS i
determined
for each guess. A new estimate of the carryovei
factor
is made by finding
the minimum
of a quadratk
function
fit to SS from three prior estimates
by a Lagrange
polynomial.
The new estimate
replaces the worst old estimate, and iterations
continue
until no significant
difference
exists
in 55,, between
the last three
estimates
(relative
difference
<0.0001).
Numerically,
this procedure
is quite
similar
to the super modified simplex algorithm
applied te
one dimension
(6). Other numerical
algorithms for minimization may be more or less effective
than the one chosea
here but were not evaluated
in this study (i.e., Fibonacci
search or Golden Section Search;
7-9).
Simulated
data samples were produced
for this Monte
Carlo study by generating
a sequence of numbers
with e
fixed ratio of means between high and low concentrations
Carryover
was then introduced
by using equation
1 above,
with the first
sample
in a data set adjusted
as if the
sequence
was being started
from a reagent
blank (zerc
sample).
Pseudo-random
variation
was then superimposed
to have a constant
variation
and a gaussian
distribution,
Both algorithms
give results similar to those shown wher
variation
is added such that it is proportional
to the meaii
(data not shown).
Carryover factors were determined
foi
every transition
between-concentrations
(except the transi
tion at the start of the sequence) by the traditional
algo.
rithm and then averaged for a carryover
factor estimate foi
the run. The carryover
factor was also determined
by the
MCSS method with the first subset of the sequence omitted,
because the transition
from reagent blank to sample is nol
equivalent
to subsequent
transitions.
One thousand
data
samples were generated
and the mean and standard
devi.
ation for the carryover
factor estimate were determined
foi
both the traditional
and MCSS
method
at each set ol
variable
parameters
(simulated carryover
1%, expressed
as percent
or carryover
factor multiplied
by 100%; coeffi.
cient of variation
1% at a mean of 100; ratio of means o
simulated standards
= 20; number
of replicates per group =
3; number
of data points per run
30; high standard
100
except as indicated).
Carryover
in a random-access
environment
is essentiall)
different,
because
the interaction
occurs
only between
th
last sample in a sequence of one method
(method
A foi
=
example)
and the
method immediately
first sample
in a sequence
of anothei
following a transition
(method B). Thi
interaction
may not be due to a difference
in concentration
of a common
analyte
as in within-method
carryover
bui
instead
due to a difference
in chemical
composition
of thi
analytical
reagents. Additional
possibilities
exist, because
reagent may produce
a simple shift up or down in the firsi
result of method
B, or it may enhance
or inhibit
th
analytical
reaction of method B, resulting
in a proportionate
error, or both. Estimation
of both within-method
and between-method
carryover
is necessary
for a random-access
analyzer
and may be performed
best by two separate
experiments.
Within-method
carryover
can be estimated
by
the MCSS
method
by reverting
to the batch mode or
continually
programming
the random-access
analyzer
to
accept samples
in a sequence as above for one method. If the
assumption
is made that the influence
between
the first
assay in method B (following
a transition
from method A)
and the second is governed by the same forces as measured
by the within-method
carryover,
then a series of duplicates
can be run by the alternative
methods and the difference
between the first-sample
result and the second, corrected for
within-method
carryover,
can be used to estimate betweenmethod carryover.
It must be remembered
that the transition from method A to B will not have the same effect as the
transition
from method B to A, so both types of carryover
must be estimated.
If such an experiment
is performed
where the concentration
of analyte
for method B is varied
through
the analytical
range, the determined
result
can be
regressed
against the estimated
true result (within-method
carryover-corrected
result
for the second sample)
and the
existence
of constant
and proportional
bias estimated
from
the significance
of the intercept
and the difference
of the
slope from 1.0(10).
With this regression equation in hand, a
true result can be estimated
by inverting
the equation
and
solving for tha expected
true result
given the determined
one. Linear regression
under these circumstances
violates
certain assumptions.
The independent
variable,
for example, is not a known fixed variable but instead has a variance.
The situation where both independent
and dependent variables have a variance
is one where the Deming
method of
regression
applies
(11). In addition,
the variance
of an
automated
analytical
method often varies with the level of
analyte. The way in which variance changes with the mean
of the independent
variable
(or dependent
variable
in this
case) influences
the precision
of regression
estimates.
Weighted
regression
corrects
for variance,
which is dependent on the mean of the independent
variable
(10).
In this study, proportional and constant
bias (simulating
method-to-method
interaction)
is introduced
in the first
sample of a pair and then the second sample is adjusted
for
within-method
carryover as described
above. This process is
repeated
until the number
of samples
is equivalent
to the
total
number
per run. Pseudo-random
variation
is then
introduced
to be either constant,
proportional
to the mean,
or half constant
and half proportional.
The first sample of a
pair in this simulated
data set is then regressed against the
within-method
carryover-corrected
second sample of a pair
using least
squares,
Deming,
weighted, and weighted
Deming regression analysis. One thousand
samples of data were
generated
and regression
estimates
for slope and intercept
(obtained
by the four methods)
were averaged
and the
standard deviation
for the estimates
calculated
for each set
of variable parameters
(coefficient
of variation
at a mean of
50
5%, range covered by standards
50, number of data
points per run
22, within-method
carryover
2%, constant between-method
carryover
5.0, proportional
error
0.9, ii standards
concentrations
centered
around 50 except
as indicated,
variance
proportional
to the mean). This would
be equivalent
to a glucose assay,
for example, with analytical range of 0 to 50 mmol/L,
and standards
run in duplicate
at concentrations
of 12.5, 15.0, 17.5, 20.0, 22.5, 25.0, 27.5,
30.0, 32.5, 35.0, and 37.5 mmolJL following duplicate
assays
=
of a second
method (calcium,
for example).
Then, in this
example,
the precision of the median standard
would be 25.0
1.25 mmol/L
(mean standard
deviation)
and proportional to the mean. The first
sample
of duplicate
standards
following
a method transition would be shifted
upwards
a
constant
2.5 mmol/L
and downwards
by 10%. The first and
last standard duplicates
would be expected to have values of
13.75, 12.525,
and 36.25, 37.475, respectively.
Results
are given in the text as mean standard
deviation for the estimate for 1000 samples
of data.
Results
Within-Method
Carryover
Estimation
When
the ratio of coefficient
of variation
to percent
carryover
is less than 2.0, the accuracy
of the MCSS
method
is greater than 99% (Figure 1, open circles).The traditional
method gives an estimate
that is consistently
low by -11%
and is less precise.
As the analytical
precision is decreased,
both methods
become biased low, with the traditional
method showing
greater
stability
at very low analytical
precision. If run conditions are changed to include 40 data points
per run of duplicates
with the ratio of high to low mean of
100, the precision
of both estimates
is improved
by 30%,
becoming
virtually
indistinguishable,
and the accuracy
of
the traditional
method is improved to -95% (Figure 1, solid
symbols).
The MCSS
method is now accurate
over a much
wider range of analytical
imprecision,
but the traditional
method becomes
biased high by 100% when the ratio of
coefficient
of variation
to percent carryover
is 10.0. Carryover estimation
by either algorithm
is very sensitive
to
analytical
precision,
a fact that must be considered when
one is attempting
to measure
this parameter.
A ratio of
standard
means in the range of 20 to 100 may appear high
but is usually
attainable
with state-of-the-art
methodology
1.5
a
U
1.0
a
a
E
a
0.5
0
a
a
Coefficient
of
Variation
CLINICAL
CHEMISTRY,
1807
and, if precision
is fairly good, is not necessary
for either
precise or accurate
estimates
of carryover.
Inaccuracy
associated with the MCSS
algorithm when precision is exceptionally
poor is attributable
to over-correction
for carryover,
which tends to decrease random variation
and hence SS.
Inaccuracy
in the traditional
method is apparently
inherent
in the assumptions
made in its derivation.
The true carryover
can be estimated
over a wide range of
expected
values with accuracy
and precision by the MCSS
algorithm
(Figure
2). The MCSS algorithm
remains
accurate to better than 99% under almost all conditions, whereas
the accuracy
of the traditional
algorithm
is roughly 89%.
Increasing
the number
of data points per run improves
the precision of these estimates
with the MCSS algorithm
improved slightly more than the traditional
algorithm
(Figure 3). In addition to improvement
of precision,
the accuracy
of the traditional
method
is continuously
improved
as the
number
of data points per run is increased.
This effect may
be due to the inherent
assumption
in the traditional
carryover
model
of an infinite
series of sample interactions,
which is better approximated
as the run length is increased.
Both
methods
are insensitive
to the ratio of means
between
standards
or samples used to estimate
carryover
when this ratio is greater than 1.1 (Figure
4). This ratio is
readily exceeded when an experiment
is set up to determine
carryover
and therefore
should not represent a significant
factor in experimental
design.
Increasing
the number of replicates
in a group while the
total
number
of data points per run is held fixed has a
deleterious
effect on the precision of estimates
generated
by
either the MOSS or traditional
algorithms (Figure 5). The
accuracy
of the MCSS algorithm
is unaffected,
whereas the
accuracy
of the traditional
algorithm
is dramatically
reduced by increasing
the number of replicates
per group.
One of the benefits
of correction
for carryover
is an
a
a.
#{149}_
1.0
a
>
0
0.8
U
0.6
1
V
aa
0.4
0.2
45
55
0.6
2
;
a
0.4
E
a
V
C
0.2
15
25
Data
35
per
Points
Sample
Fig. 3. Effect of the number of data points per run on estimating withinmethod carryover
Mean (A) and standard deviation ( for percent carryover estimated wth all other
conditions as indicated in Materials and Methods. Symbols for Figs. 3-5 as in Fig.
2
improved estimate
of the true within-group
variance when
data of more than one level are submitted
to analysis
of
variance
for the evaluation
of the precision of a methodology. For example, when the true coefficient of variation
is 1%
(mean
100.0) and carryover
is 10% (ratio of means
20,
30 data points per run and three repeats per group),
the
estimated
within-group
coefficient
of variation
is 5.3 1%
before carryover
correction
and 1.08% after MCSS carryover
=
1.0
I
1.0
-----------
0
a
E
-,
0.8
a
U
Ii
a 0.9
a
a
0.6
0
a
a
E
4IlIIIlIllI
0.4
a
UI
0.2
0.48
0
a
0.46
5
C
0
0.44
a
a
a
o.ss
a
0
10
I,
Actusi
Carryover
1808
CLINICAL CHEMISTRY,
20
40
Ratio
of
60
Simple
80
100
Means
0
1.0
a
a
UI
a.
a
C
0.9
0.8
0
0.
0
a.
I
V
a
U
E
a
a
a
0.7
UI
a
UI
0.6
a
a
0
0.4
V
V
a
(0
0.2
(0
6
0
Repllcataa
p.r
correction.
Little effect is seen if carryover
is very small as
compared
with analytical
imprecision. For example,
if the
true coefficient
of variation
is 1% and carryover
is 0.1%, the
estimated
coefficients
of variation
are 1.00%
and 0.98%
before and after MCSS carryover
correction.
The traditional
method for carryover
factor estimation
is
noniterative,
whereas
the MCSS method is iterative,
leading to concern
for a potential
increase
in processing
time.
During
this study, >99%
of the data sets analyzed
converged within
20 iterations
with a total processing
time less
than
2 s for the case of 30 data points
per run, three data
point per group. It is possible that this performance
could be
further
improved
by use of one of the other numerical
minimization
methods referenced in Materials
and Methods.
Carryover
Estimation
first-sample
secondsample result, is an estimate of proportional
error caused
by
between-method
carryover.
When the slope is significantly
less than one, the negative
error is indicative
of an inhibition of the analytical
reaction. The intercept is the estimate
of constant
error. Because
the estimates
of slope and intercept are negatively
correlated,
when the accuracy
of the
slope is poor then the intercept
is also inaccurately
estimated. Precisions
of estimating
slope and intercept
are also
related,
and when the slope is poorly determined,
so also will
be the intercept.
Thus the results are presented
in terms of
the accuracy
and precision of the slope estimate and will be
considered
to apply as well to the estimate
of the intercept.
Further
information
on the statistics
of estimating
both
intercept
and slope are available
upon request.
When variation
is constant,
the Deming
regression
estimate for the slope is very accurate
(m
slope
0.905
0.181
for a true
M
0.9),
whereas
the least-squares
estimate
is biased low (m = 0.790
0.147),
and weighting
The
result
slope,
determined
on the within-method
by
regressing
the
carryover-corrected
has
no effect
on accuracy
or precision.
10
Coefficient
method
Between-Method
15
20
Group
When
variation
is
of
variation
proportional
to the mean of the independent variable,
however, the Deming
regression
estimate
for slope is biased
slightly
high (0.916 0.201) and is improved by weighting
(0.905 0.183). The least-squares
estimate
becomes
less
precise (0.788 0.162) and is improved by weighting
(0.816
0.152)
when variation
is proportional
to the mean.
When
variation
is partly
proportional
and partly
constant,
the
results
are
Increasing
between
those
presented.
the coefficient
of variation
beyond
5.0 increases the low bias of the slope estimated
by least squares
and weighted
least squares and introduces
a high bias in the
Deming
and weighted Deming slope estimates
(Figure
6A).
The Deming
and weighted Deming are still more accurate
than least squares
and weighted
least
squares
until
a
coefficient of variation
of 20.0 is reached, when the Deming
estimates
become
totally unreliable.
The precision of the
Deming
and weighted Deming estimates is less than that of
least squares
and weighted
least
squares
(Figure
6B).
The analytical
range covered by the samples
used for
estimation of constant
and proportional
error is an important determinant
of the accuracy and precision
of the
resulting
estimates
(Figure
7A). As the covered analytical
range
decreases
below 20 (all other conditions
as stated in
methods),
the low bias of the least squares
and weighted
least-squares
estimates
increases
and the Deming
and
weighted Deming
estimates
become biased high. The precision of these estimates
becomes
less as the covered
analytical range becomes
less than 60 (Figure 7B). The covered
analytical
range
required
for accurately
estimating
the
slope is inversely related
to the analytical
precision, and, as
the precision improves,
the lower is the range
of samples
required
for accurate
and precise
estimates
(accurate
and
precise
estimates
obtained whenever
covered
analytical
range/coefficient
of variation
>10 in this study).
Constant
error can be accurately
and precisely
estimated
CLINICAL CHEMISTRY,
1809
UI
0.
0
0.2
V
a
El
a
UI
40
C
0
;30
20
1o
20
Range
40
In
60
D.pend.nt/Ind.p.nd.nt
80
v.rlabls
through
constant
1810
estimates
and significantly
precision
improved
the
of estimation.
(A)
Deming
Discussion
Mean
weighted
a wide
CLINICAL
CHEMISTRY,
0
UI
0.90
0
0.
0
a.
0.85
a
a
UI
0
0.1
a
0
V
0.1
a
(0
10
20
Number
30
of Data
40
Point.
50
per
60
Run
Symbols
as in Fig. 6
(or independent
variable,
to
the mean.
Since the assumption
can be made that the ratio
of variances
is equal to one, the Deming
regression
method
can be applied
without
any additional
requirement
for the
determination
of variances.
This
may lead to only slight
inaccuracy
unweighted
at
very
large proportional
errors
when the
procedure
is applied and the variation
in the
(or independent
variable)
is proportional
to the
dependent
mean.
The application
of weighted
regression
requires
estimating either a variance
at each concentration
of analyte
or the
determination
of an equation
that
fits the relationship
of
variance
to the mean. The variance
was known
in this study
so that weighted
regression
could be applied without
the
independent
determination of a variance-vs-mean
relationship. The determination
of variance
at each value could be a
tedious
task; however,
this is usually
required
in the evaluation of an analytical
method. The additional
requirement
of
determining
how the variation
varies with the mean would
also be useful in the evaluation
of a method, because
this
information
is actually
necessary
for any statistical
comparison of means,
If the variance
is not very dependent
on the
mean,
then weighted
regression
is unnecessary
and probably undesirable.
The combination
of withinand between-method
carryover estimation
should
make it possible
to predict
the
degree of this type of error in most analytical
situations.
Online carryover
correction
in an automated
analyzer
could be
used to improve the aiuracy
of a result or to flag a result for
which accuracy
is questionable.
Such correction
should, in
addition,
improve the precision
of an analytical
result and
enhance
the ability
of a method to discriminate
concentrations
of analytical
samples.
between
References
1. Broughton
PMG,
Buttoiph
MA, Gowenlock
AH, Neill DW,
Skentelbery RG. A guide to automation in clinical chemistry.
J Clin
Pathol 1969;22:278-85.
2. Dixon K. A theoretical
study of carryover
in discrete and
continuous-flow
systems. Ann Clin Biochem 1982;19:224-6.
3. Moses GC, Lightle
GO, Tuckerman
JF, Henderson
AR. The
EPOS Automated
Selective
Chemistry
Analyzer
evaluated.
Clin
Chem 1986;32:165-9.
4, Nolan JP, DiBenedetto
B, Tarsa NJ. Continuous-flow enzyme
immunoassay
for thyroxin
in serum.
Clin Chem 1981;27:738-41.
5. Broughton PMG. Carry-over
in automatic analyzers. J Automated Chem 1984;6:94-5.
6. Routh MW, Swartz
PA, Denton MB. Performance
of the super
modified
simplex.
Anal
Chem
1977;49:1422-8.
of optimization.
New
analysis.
York:
York:
methJohn
11. Cornbleet
PJ, Gochman
N. Incorrect
least-squares
regression
coefficients
in method
comparison
analysis.
Clin Chem
1979;25:432-8.
12. Knedel M, Haeckel R, Seidel D, eta!. Analytical
performance
of
the random
access Hitachi
737: a multicentre
evaluation.
J Clin
Chem Clin Biochem 1986;24:409-32.
CLINICAL
CHEMISTRY,
1811