Quantifying Probabilistic Expressions

Quantifying Probabilistic Expressions
Author(s): Frederick Mosteller and Cleo Youtz

Source: Statistical Science, Vol. 5, No. 1 (Feb., 1990), pp. 2-12
Published by: Institute of Mathematical Statistics
Stable URL: http://www.jstor.org/stable/2245869 .
Accessed: 27/12/2013 20:49
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp
.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.
Institute of Mathematical Statistics is collaborating with JSTOR to digitize, preserve and extend access to
Statistical Science.
http://www.jstor.org
This content downloaded from 171.67.34.205 on Fri, 27 Dec 2013 20:49:43 PM

All use subject to JSTOR Terms and Conditions
StatisticalScience
1990,Vol. 5, No. 1, 2-34
Quantifying
Probabilistic
Expressions
FrederickMostellerand Cleo Youtz
Abstract.For 20 different
studies,Table 1 tabulatesnumericalaveragesof
opinionson quantitative
meaningsof 52 qualitativeprobabilistic
expressions.Populationswithdiffering
occupations,
mainlystudents,
physicians,
othermedicalworkers,and sciencewriters,contributed.
In spite of the
varietyof populations,formatof question,instructions,
and context,the
variationoftheaveragesformostoftheexpressions
was modest,suggesting
that theymightbe usefulforcodification.
One exceptionwas possible,
becauseit had distinctly
different
meaningsfordifferent
people.We report
newdata froma surveyofsciencewriters.
The effectofmodifiers
suchas
veryor negation(not,un-,im-,in-) can be describedapproximately
by a
simplerule.The modifiedexpressionhas probability
meaninghalfas far
fromtheappropriate
boundary(0 or 100) as thatoftheoriginalexpression.
This paperalso reviewsstudiesthatshowstabilityofmeaningsover20
years,mildeffects
oftranslation
intootherlanguages,context,smallorder
and effects
ofscale forreporting
on extremevalues.
effects,
The stemprobability
withmodifiers
givesa substantialrange6% to 91%
andthestemchancemightdo as welliftriedwithvery.The stemsfrequent,
probable,
likely,
and oftenwithmodifiers
produceroughly
equivalentsetsof
means,butdo notcoveras widea rangeas probability.
Extremevaluessuch
as alwaysand certainfallat 98% and 95%,respectively,
and impossible
and
neverat 1%.
The nextstepwillbe to offer
codifications
andsee howsatisfactory
people
findthem.
Keywordsandphrases:Quantifying
language,codifying
language,meaning
ofqualitativeexpressions.
INTRODUCTION
misunderstood,
whetherthe emitters
or receiversare
physicians,patients,scientists,science writers,or
In everyday
language,peopleapplytheexpressions
othercitizens.By associatingnumericalvalues with
alwaysand certainto eventsthatoccurin fewerthan
specificqualitativeexpressions
we mayultimately
im100%oftheiropportunities;
furthermore,
on average,
provecommunication.
In the long run we plan to
peopleregardveryhighprobability
as morelikelythan
proposesomecodification.
In thispaper,we wantto
almostcertain,
a surpriseto manyofus. Communicareportwhatsuchexpressions
currently
meanbysumtionsthatemployqualitativeexpressions
forfrequenmarizingresultsfrommanystudiesand givingnew
cies or rates of occurrencerun the risk of being
findings
fromsciencewriters.
In this firsttreatment,
our intendedreadersand
users are scientistsincludingstatisticians,
because
Frederick
Mostelleris RogerL Lee Professor
ofMaththeirlanguageoftenneeds interpretation
to wider
ematicalStatistics,Emeritusat Harvard University audiencesthan specialists.Statisticianshave special
and Directorofthe Technology
Assessment
Groupin
skills,interests,
and stakesin communication
of inthe HarvardSchoolof PublicHealth. Cleo Youtz is
formation
aboutprobabilities.
Theymayalso wishto
Mathematical
Assistantin theDepartment
ofStatistics participatein producing
a codification
ofprobability
at HarvardUniversity.
expressions.
As thisworkis stillinprogress,
have twomainforms.First,in some
readersare encourCodifications
aged to send commentson the paper to Frederick areasofworka fewstandardexpressions
maybe used
at theDepartment
for
all
Mosteller
to
of
belief
or relative
situations
express
degree
ofStatistics,
ScienceCenFor
Kent
codified
some
frequency.
ter,1 Oxford
example
[33]
Street,Cambridge,
Massachusetts
02138.
2

QUANTIFYING PROBABILISTIC EXPRESSIONS
in theintelligence
fieldin termsofodds.
expressions
He used as a base almostcertain,chancesare good,
and chancesaboutevenand quotedoddsofat least9:1
in favor,at least 3:1 in favor,and 1:1 or a 50-50
withthereverseoddsgoingwith
chance,respectively,
not hapabout something
statements
corresponding
seem
pening.Whetherwe-likeit or not,statisticians
thatassociates
a similarcodification
tohavegenerated
significant
numbersless than 0.05 withstatistically
and numbersgreaterthan 0.05 withnotstatistically
significant.
wouldmerelyrelate
A secondformof codification
sets of probability
expressionsto certainrangesof
numbersso as to offersomewhatmoreprecisionin
Inforlanguageto thosewhowishto use thefindings.
mationin the presentpaper could be used forthis
dishereinvitespreliminary
purpose.Its presentation
thatcouldbe the basis foradcussionand criticism
ditional work before firmingup either form of
codification.
Manypeoplesaythatone cannotputa singlenumberon a qualitativeword.Actuallyone can putmany

numbers
on a qualitativeword,and thatis one reason
forpursuingsuch studies.For the currentstudy,we
selected52 expressionsfroman initiallist of about
300. Most of these52 expressionshave been studied
Table 1 displaysthe average
by otherinvestigators.
probabilityto the nearestwhole percentforthese
expressionsgivenin 19 otherstudies[1-19] and the
currentstudy[20] of sciencewriters.Our references
[1-201at the end ofthispaperalso includeinformation about sample sizes and kinds of respondents
in each study.As one mighthope,the
participating
studiesgivesimilar,thoughnot identical,resultsfor
whensamplingand othersources
thesameexpression
ofvariability
are considered.
The right-hand
side of Table 1 givesthe average
in 20 studies,with
reported
byrespondents
probability
subscriptsindicatingthe studiesor partsof studies
reporting
a givenpercentage.Because some studies
or samplesfor
usedmorethanone setofinstructions
we oftenlist morethan one rea givenexpression,
sponsefora study.In additionto the specificresult
foreach study,Table 1 also givestwoaveragesacross
the studiesand partsof studies.The firstaverageis
the unweighted(equallyweighted)averageforthe
studies,and thesecondgivesthegrandaveragewhen
averagesforthe studiesare weightedby the number
ofrespondents.
The choiceofaveragedoes notmake
in thesummary
ofstudies.
muchdifference
TWOEXAMPLES
rarein a medical
Mapes [11]studiedtheexpression
to assesstheprobability
context.He askedphysicians
and an
ofsideeffects
fromtwodrugs-a beta-blocker
59.4%ofthephyForthebeta-blocker
antihistamine.
"lessthan1 perthousand,"
sicianschosethecategory
foran antiwhereas20.7 percentchosethiscategory
arises
thatthedifference
Mapes suggested
histamine.
are
because the side effectsfroman antihistamine
His
muchmildercomparedto thoseof,abeta-blocker.
pointis thatthemeaningofrarechangeswithcontext.
On the otherhand, physiciansmay actuallyhave
ofthe
different
oftherateofside effects
perceptions
twotypesofmedication.
PresidentGerald R. Ford said that a swine flu
epidemicin the 1976-1977season was "a veryreal
estimatesfrom
[23]gotprobability
Boffey
possibility."
fourexperts:2%, 10%, 35%, and "less than even."
Boffeyconcludedfromthese estimatesthat the
chancesoftheepidemicare"farlowerthantheofficial
rhetoric... would lead one to expect." Thus he sees a
betweenthemeaningofthe qualitative
disagreement
and an averageof
expression"veryreal probability"
perhaps20 to 25%. Mosteller[28]useddatafromCliff
[24] and Selvidge[17] to estimatea publicmeaning
at 29%.
of"veryrealpossibility"
thevariationin perThese twoexamplesillustrate
expressions.
ceptionofthemeaningofprobability
CONTEXT
It has oftenbeen pointedout that the context
the
counts,rates,oramountscan influence
employing
estimate.For an examplementionedto one of the
authorsby Leo Crespi,"a handfulof grapes"and "a
handfulofpeopleon thebeachat ConeyIsland"could
the firstin the range
estimates,
implyverydifferent
of 5 to 20 grapes,and the secondhundredsor even
ofcourse,complithousandsofpeople.This problem,
We think
expressions.
catesmattersforprobabilistic
ofthestudiesdiscussedhereas dealingwithprobabilfor
itiesin theregionfrom1 to 99%. The probabilities
veryrareevents,suchas atomicdisastersand regional
blackoutsfromfailureofthe powersystem,are very
to laymen;and,as weunderto communicate
difficult
stand,muchremainsto be learned.Communicating
problem
the idea ofverysmallrisksis an important
in theriskfield;it deservesand has its ownresearch
and Lichtenstein
[31]).
effort
(Slovic,Fischoff
Some authors have emphasizedthe differences
thatchangingcontextcan create.For example,in a
of propositions,
Tverskyand
studyof formulation
Kahneman [32] show verysubstantialchangesin
andan equivalentgambetweena certainty
preference
in termsof
the samepossibilities
ble by formulating
lives saved versuslives lost. Kahneman,Slovic and
Tversky[27]includearticlesbymanyauthorsshowing
effects
ofcontext.Ouremphasisis moreon the near
in Table 1,rather
ofopinionsas illustrated
constancy
side of
thoughthe right-hand
than the differences,

F. MOSTELLER AND C. YOUTZ

TABLE 1
Averageprobabilitiesexpressedin percentagesfor52 expressionsfrom20 studies
Unweighted Weighted
average
average
Unweighted Weighted
average
average
98
99
38
38
Less oftenthan not
15
14
Notoften
13
13
Not veryoften
9814*,20
42
37
Possible
78& 828b 9?&,13*,20
78
81
High chance
58
58
50
50
Betterthan even chance

5820
Even chance
41
41
Less than an even chance
720
14
13
Poor chance
9017a* 9117t*,20
15
13
Low chance
66
66
Liable to happen
36
36
6620
Might happen
77
79
Usually
Always
965 9715a,
941,2
91
91
Almostalways
8919
885
95
97
81
86
81
947
9316a
1320
951,&
Almostcertain
61
Veryfrequent
Frequent
45
45
Notinfrequent
17
17
Infrequent
4520
91
84
91
81
1720
52
Veryinfrequent
4120
Veryhighprobability
16
High probability
1020 2013*
Moderate probability
Low probability
171
1620
Verylow probability
620
82
85
Verylikely
7910
746
69
69
Likely
17
16
Unlikely
616
879
858a,13*,2o
638b,8c
7013*,20
65lo
148b
Veryunlikely
82
85
Veryprobable
1513*
70
69
Probable
16
15
626 64& 658b

Improbable
620 99 1013* 1410
19
Unusually
28
26
Sometimes
87
729 731 742 8014*
1915b
1720
189 201,2,14*
65
69
17
Once in a while
39
37
Not unreasonable
22
22
Occasionally
328b
8013* 879,20
6212a
3920
7013*,20
2718b*
282o 293*
3816b
2112a, 12b,15a, 19,2o 2318b* 244b
719, 10 722 771 8014*
26
23
Now and then
172
12
12
Seldom
Veryseldom
Rarely
182o 2018* 2518b* 325 343*
1513* 166,20
181o 2014*
715b 816a 93* 104a,18a*,18b* 1320
6,8*,18b*,20
873* 8818a*,18b*
571rc
6412b
5915b,
704a,20
16a,
7116a
73*
162,9
19
6017a*
732,18b*
2316
617,16b
743* 755
Veryrarely
61
More oftenthan not
Almostnever
50
50
As oftenas not
6218b*
647
Never
23*,8d
3,8*,18b*
420
75
02, 3*,4a,4b, 7, 1&a*,19,20 15,a, 15b, 18b* 3d
instructions
or samplesizes.
list,a, b, c andd indicatedifferent
The subscripts
indicatestudieslistedin thereference
* Indicatesmedian.

185
126
420
62
, ab?20
8018b*
471
1715b 207,18a*
7818a*
506
3716c
53*, 7,1a, 18a*,18b*,19 720 82 915b 1216b
5017b*
5920
793*
283*,4a
Very often -
Often
2212b,15a
2012a,,8a*
19
156
Veryimprobable
824b,2o
767 779 782
7420
1920
513* 620 1 10
85
7119
8518a*
15,8* 16,8b* 1820 223* 245
11
138b
7015b
8415a
321 337 3416a 364c
1017a*,17b*
129
3620
19
236 2510
796,10
1410
1320
5120 611
16
114*20
5013*,20
1912b
8020 871
56
Impossible
721o 8013*8220
568b 6612b 6712a,20
3611b 3911a
1612a
3320 379 382 4013*431 516 5510,14*
278b
8120
55
13,8* 155 163* 1920
1118b*
9015b 9120
Certain
91lo
86
16c, 19
1003*,,4a,4b,4c,
3820
9816a, 16b 997, 18a*, 18b*,20
1b
1516a
Table 1 illustratesdifferences
owingto samples,inor context.
structions
In comparingcontextresultswithno-context(in
isolation)resultsfor 22 expressions,Selvidge [17]
roundedto the nearest5% for
foundthatdifferences
5 for7 expressions,
medianswere0 for10expressions,
and 15 for1 expression.The
10 for4 expression's,
wasforappreciably,
whichwentfrom
largestdifference
25% in isolationto 10% in context.
Pepperand Prytulak[30] offeran elaborateinvestigationfor studyingeffectof contextby creating
andhigh-frequency
forframlow-,moderatecontexts
ing the same expressions,as well as evaluationin
isolation.The main sourceof divergencewas that
in thecontextofairplanecrashesand of
probabilities
ofearthquakes
werefarremovedfrom
theoccurrence
the numericalvalues giventhe same expressionsin
remarks
aboutstudentsmissingbreakfasts
ortheproportionofmenthatMiss Swedenthoughtfoundher
The paperillustrates
thatcontextcan push
attractive.
events
themeaninga goodway,butthatforordinary
are modest.We emphasizeagainthat
thedifferences
we are notconsidering
numerical
evaluationsofprobabilisticexpressionsin such contextsas veryrare
in shorttimeperiods.Contexttakes
eventsoccurring
severalformsin additionto the substantivetopic,
suchas typeof scale used and orderofpresentation.
in the sectionon
We discussthese mattersfurther
specialtopics.
SCIENCE WRITERS
We gathered
data on meaningsofthe52 qualitative
fromsciencewriters
expressions
through
a mailquesFor a situationwithoutcontext,
tionnaire.
thewriters
gaveestimatesforprobabilities
forthe52 expressions
and lowerand upperlimitstheythought
theirreaders
wouldset foreach expression.
The responseratewas
about37%. The averageresponsesare similarto those
fromotherstudiesreportedin Table 1. The relation
oftherangesreported
fortheestimatesis informative
in spiteof the low responserate. Some background
for the special study of the science writersand
the selectionof the 52 expressionsappears in the
Appendix.
RESULTS
Median, Quartiles and Variability
In Table 2 we givethemedian,quartiles,and interofthesciencewritquartilerangeforthedistribution

ers' own point estimatesfor each expression.We
districomputedmediansand quartilesforfrequency
butionsfromFormA and FormB (see Appendix)and
thenaveragedresultsforthetwoforms.
As oneexpects,extreme
expressions
likealwaysand
neverhave smallvariationas measuredbythe interquartilerange(IQR), the distancebetweenthe quartilesof the cumulativedistribution.
As a measureof
variability
of freguency
distributions
of the science
writers'"ownestimates,"
we use the IQR, becauseit
is lesssensitive
thanthestandarddeviation
toextreme
values. More centrallylocated expressions(nearer
50%) usuallyhavebroadervariation.
Examples of Distributions
Figure1 showsrelativefrequency
distributions
(using class intervalsof length10 centeredon certain
of5) forthesciencewriters'
multiples
personalchoices
forpossible,almostalways,and unlikely.We chose
theseexpressionsto give a notionof the varietyof
distributions
To get a feelingforthe
encountered.
behaviorand systematic
movement
ofdistributions
as
the modifiers
fora stemchange,runningdownthe
medianand quartilesin Table 2 fora givenstem
quicklyshowshowthemiddle50% ofthedistribution
moves.Figure1 showsthatpossiblehas a bimodal
distribution.
This bimodalityalreadysuggeststhat
as a qualitativeexpression.
possibleis unsatisfactory
We returnto it below.Otherexpressionswerenot
bimodal(exceptforsuch minorvariationsfora few
as expectedin histograms).
expressions
Relation of Variabilityto Level
Figure2 showsa plotoftheIQR againstthemedian

M. Because of anchoringat the end-points,
we expect the IQR to rise as the medianmovesfromthe
extremestoward50%, and thus we expect a cap
wellintheintervals
shape.Thisideais borneoutfairly
0 < M c 33 and 67 c M c 100,butnotin thecentral
interval33 < M < 67,as we nowdiscuss.
The Middle Expressions
The 11 expressionswithmedianprobabilities
between33 and 67 sortthemselvesinto groupswhose
differs.
variability
Group1 containsnumbersthateithercomparedirectlywiththe 50-50 situation(less than an even
chance,evenchance,betterthanevenchance)or compare the chanceof the eventwiththat of the nonevent(less oftenthannot,as oftenas not,moreoften
lead to smallinterquarthannot).These expressions
tile ranges.The six dots at the base of the middle
them.Essentially
thevalue
panelofFigure2 represent
50% is anchoring
the responses,especiallywitheven
chanceand as oftenas not. We mightget similar
anchoringeffectselsewhereif we asked how often
"aboutoncein n times"occurred.
respondents
thought
offer
andarethereGroup1 expressions
highprecision
for
forepotentially
codification.
attractive

F. MOSTELLER
AND C. YOUTZ
TABLE 2
Quartiles,median and interquartilerangeforthe sciencewriters'ownpreferredestimatesfor52 probabilityexpressions,pooledfrom
producedbyFormA and Form B
distributions
25%
Median
75%
IQR
10.3
5.3
19.7
10.1
24.8
19.6
14.5
14.3
7.5
.2
38.5
.3
50.2
.5
42.7
.3
High chance
Betterthan even chance
Even chance
Less than an even chance
Poor chance
77.5
53.3
49.7
39.6
8.4
80.4
57.6
50.0
40.2
10.3
89.1
60.2
50.2
45.0
19.7
11.7
6.9
.5
5.4
11.3
Low chance
Liable to happen
Mighthappen
5.0
59.8
19.9
9.8
68.2
37.6
12.8
77.7
50.1
7.8
17.9
30.2
Usually
65.6
75.1
82.2
16.7
9.9
17.5
9.9
23.5
17.4
25.0
15.3
37.6
26.1
35.0
22.4
52.6
16.3
17.5
12.5
29.1
12.5
9.8
7.4
3.2
20.0
15.1
10.2
4.9
27.7
25.0
17.5
7.7
15.2
15.1
10.1
4.5
Expression
25%
Median
75%
IQR
Always
Almostalways
99.6
89.7
99.7
91.7
99.8
95.2
.3
5.5
Not often
Not veryoften
Certain
Almostcertain
98.7
87.5
99.6
90.2
99.8
95.0
1.1
7.5
Possible
Impossible
Veryfrequent
Frequent
Not infrequent
Infrequent
Veryinfrequent
75.3
60.0
32.7
10.1
3.6
82.6
72.2
49.6
17.3
5.2
89.7
75.3
57.3
22.6
10.0
14.5
15.2
24.6
12.5
6.4
Veryhighprobability
High probability
Moderateprobability
Low probability
Verylow probability
89.8
77.1
40.1
7.8
1.9
92.5
82.3
52.4
15.0
4.9
95.2
87.2
58.7
22.3
7.6
5.4
10.1
18.5
14.5
5.7
Expression
Verylikely
Likely
Unlikely
Veryunlikely
80.1
62.6
9.8
2.7
87.5
71.1
17.2
5.0
90.2
77.6
22.7
9.8
10.1
15.0
13.0
7.1
Unusually
Sometimes
Once in a while
Not unreasonable
Veryprobable
Probable
Improbable
Veryimprobable
81.5
64.7
7.6
1.5
89.7
70.2
12.5
4.8
90.4
77.7
22.3
7.5
8.9
13.0
14.7
5.9
Occasionally
Now and then
Seldom
Veryseldom
Veryoften
Often
More oftenthan not
As oftenas not
Less oftenthan not
77.5
65.0
57.1
49.8
34.8
82.8
72.5
59.8
50.0
40.0
89.9
75.4
60.4
50.3
42.7
12.4
10.4
3.3
.6
7.9
Rarely
Veryrarely
3.6
1.2
7.2
3.0
10.0
5.0
6.5
3.8
Almostnever
Never
1.2
.1
2.9
.3
4.6
.4
3.4
.3
50-
40~
Almost
Always
~40-
Unlikely
20
Possible
20-
30
60X0i
IQ
I *Possible
4C-
30
0
A...
/4S
15 25 35 45 55 65 75 85 95
Estimate
(percenft
ofsciencewriters'own estimatesfor
FIG. 1. Frequencydistributions
threeexpressions,gouped by 10% intervalscenteredat multiplesof
5%.
has
moderate
probability)
Group2 (not infrequentt,
an IQR aboutthesize one wouldexpectfromextrapolationfromthetwoouterthirdsofFigure2.
fromnot
extravariation,increasing
Group3 offers
and mighthappentopossible.Becauseit
unreasonable
70
MediG17
100
Plot ofinterquartilerange versusmedian ofsciencewriters'

own estimatesforthe 52 expressions.
FIG. 2.
expression,
seemeda logicalratherthana quantitative
we did notregardthefirstoftheseas a probabilistic
norwas mighthappena cheerfulchoice,
expression,
becauseofthe
buttheywereincludedforexamination
intheinterval33 to 67.Finally
ofexpressions
scarcity
possibleseemsexcellent,at firstblush,formakinga
from
but recallthebimodality
veryvaguestatement,
Figure1.

The doublenegativein not unreasonable

and not
infrequent
makesthemunattractive
forcodification,
and we will see undesirablequantitativefeaturesof
themin thenextsection.
AcceptabilityFunctions forthe Range Data
As theystand,these graphsare not relativefrequencydistributions.

To obtainmediansand quartiles
forthesedistributions,
we firststandardized
themso
that the total mass associatedwiththe percentsis
100. Then we treatedthe standardizedresultsas if
theywere distributions
and obtainedmediansand
quartilesforthegraphsforFormA and FormB, and
finallyaveragedtheresultsforthetwoformsto geta
singlepooledestimateforthequartilesand medianof
thecombinedresult.
It is usefulto knowhowtalltheoriginalgraphsare:
thatis, whatis an expression'smodalacceptability.
To smoothawaylocal fluctuations
and numberpreferences,wewanteda measureforan interval,
notjust
thepercentwiththehighestacceptability.
To measure
this,we lookedat intervals
oflength6% betweenand
includingadjacentmultiplesof 5. For mostexpres-
AlthoughTable 2 and Figure2 givean idea ofthe

central value and variabilityamong writersof
the sciencewriters'personalestimates,theydo not
makeallowanceforthe variability
(the ranges)that
Therefore
we
thewriters
reportfortheseexpressions.
introduce
a notionofan acceptability
function(used
in Reagan,Mostellerand Youtz[13]) foreachexpression,rangingoverthe possiblepercents0 to 100 as
at a givenpercentis
before.The valueofthefunction
theproportion
of respondents
whoincludethatpercentin theirrangeforthe expression.(We use prosions we used the intervals5-10, 10-15, ..., 90-95.
ofrespondents
and
portionin speakingofthefraction
We averagedthe acceptability
proportion
forthe six
of the expressionto dispercentforthe probability
in an intervaland tookas ourmeasureof
percentages
tinguishmore readilybetweentwo measuresthat
modalacceptability
thelargestaverageoftheproporcould both be called percentagesor both be called
tionsamongthe sets. Near the extremesit seemed
proportions.)
betterto givetheaveragefor1-5 orfor95-99in a few
Although
rangesofestimatesforprobability
expresinstances(namely,veryrarelyand almostnever);just
sionshavebeen gatheredbefore[1,6,7,9,16],
we anathe value at 0 or 100 foralways,never,certain,and
lyze them in more detail than usual. For each
and the value at 50 forevenchance.We
impossible;
therespondent
expression,
gavea lowerboundand an
did thisforFormA and FormB and thenaveraged
upperboundthatestimatedthe extremesforreaders
themodalacceptabilities.
oftherespondent's
work.To combinetheseto givea
When plottedagainstthe median,the modal acnotionofoverallacceptability
we
foranyexpression,
ceptability(not shown) looks like an upside-down
foreachwholepercentage
from0 to 100the
computed
versionofFigure2. Similarly,
a plotoftheIQR ofthe
ofrespondents
proportion
whoincludedthatpercent- acceptability
functions
againsttheirmediansgave a
This produced picture(notshown)likeFigure2.
age in the intervalforthe expression.
graphs-wegivepointsonlyat multiples
of5%-such
Some expressionswith exceptionally
high modal
as Figure3. The graphthenrepresents
an acceptabil- acceptability
or exceptionally
highIQR are shownin
ityfunction,
and it showsthe regionwhererespond- Table 3, together
withtheirmodalacceptability
and
theirbeliefin the appropriateness their IQR. The expressionswith highestmodal
ents concentrate
fora givenexpression.The graphalso showshow
theirbeliefis by the
strongand how concentrated
ofthefigure.
heightand narrowness
TABLE 3
Expressionswithunusuallylargeacceptabilitymodeor largeIQR
1.00
.80
o
S5
Acceptability
VeryUnlikey
~.60
.2020
40
Mode
IQR
Even chance
Always
Never
Impossible
Certain
50
96
2
4
95
99.4
98
97
93
92
5.9
6.3
5.4
6.9
8.1
Sometimes
31
71
64
47
44
51
42
mode
Largeacceptability
Likely
.40
Median
6p
80
in Percent x
Probabllity
100
FIG. 3. Acceptabilityfunctionsfor likely and very unlikely. The

heightofthe acceptability
functionat x is theproportionof respondents who includethepercentagex in theirinterval.
LargeIQR
Liableto happen
Notinfrequent
Notunreasonable
Might happen
Possible

43
47
58
51
55
24.1
25.6
27.2
31.7
35.0
42.4
are thosethatare wellanchoredeither

acceptability
in themiddleor at theextremes:
evenchance,always,
certain,impossible,
and never.The largestIQR goes
withpossible,and mighthappenis the nearestcompetitor.The lowestmodalacceptability
valuesgowith
not unreasonable,
and
mighthappen,not infrequent,
possible;we notethatthisset includesthetwodouble
negatives.
We studiedpossiblemoredeeplyby plottinga respondent'srangeforpossibleagainstthe midrange
(averageof the respondent's
bounds).The resulting
forpossibleshowthat not onlydo many
clusterings
respondents
feelthatpossiblecorresponds
toanynumberbetween0 and 100 (a reasonableliteralinterpretation),but othersassociateit withan eventso rare
thatit can scarcelyoccur(anotherreasonableinteras in barelypossible),whereasa fewassign
pretation,
it a 50-50chanceexactly(somewhata surprise),
othersabout25percentwitha largerange,andstillothers
are scatteredall overthelot.Such distinctclustering
seems to say that interpretations
of possiblediffer
and thatpossiblehas distinctmeanings
substantially
fordifferent
Thus
groups-atleastat a givenmoment.
it does not have a homogeneousmeaningof broad
range,and so its use maymisleadsomereaders,and
whatlookedgoodforvagueness"at firstblush"has
not survivedcloser examination.We made rangeversus-midrange
plots for all the expressionsbut
foundlittlenotableabouttheothers.
Earlierand Later Responses
Withlargenon-response,
it is customary
in sample
to checkforchangesinresponsebetweenearly
surveys
and late respondents(Mosteller [29], page 215;
Bartholomew[21]; and Hansen and Hurwitz[26]).
For each expression,
we comparedtheaverageprobabeforethesecondmailbilityforthosewhoresponded
forthoseresponding
ingwiththeaverageprobability
afterward.
To look for some systematictrend,we
dividedexpressions
intohigh(averageprobability
over
50%) and low(therest).Thenwe scoredeach expressionas going"up"or going"down"fromearlyto late.
2 x 2 tableexhibiteda slighttendency
The resulting
for"high"expressions
to go "down"and "low"expressions to go "up." A chi-squaredtest correctedfor
fellalmostexactlyat the 0.05 level,3.86
continuity
insteadofthetabled3.84.Although
thismightsound
like a regressioneffect,the grouping"high"versus
"low"is chosenon thebasisofall respondents,
and so
theselectionexplanationwouldnotbe veryrelevant.
The averagereductionforthe "high"groupis 0.4%,
and theaverageincreaseforthe "low"groupis 0.5%.
Roughcalculationssuggestthat 0.4% is about one
standarddeviationfortheaverageofeithertheupper
or lowerhalfof the expressions,
and so the changes
observedare comparableto thosethatsamplingfluc-
tuationswouldsuggest.The absolutesize does not

seemsubstantial,
andso thehopeis thatnon-response
is nothighly
relatedtotheprobabilities
sciencewriters
attachto expressions.
SPECIAL TOPICS
Effectof Modifiersand Prefixes
Cliff[24] in a studyofthemeaningsofexpressions
in a largercontextfoundevidencethat ratingsof
expressions
weremodifiedin an approximately
multiplicativefashionwhenadverbswereintroduced
to
modify
adjectives.The ratingsfellon a scale havinga
neutralzeropointand runningfromnegative(unfavorable)numbersthroughpositive(favorable).For
example,verywickedhad a rating1.25 timesthatof
wicked.Thus the adverbsprovidedmultipliersobtainedby averagingthe effectsoverthe expressions
wheretheadverbswereused.
In a similarspirit,we want to providesummary
valuesoftheeffects
ofmodifiers
usedinthe52 expressions,thoughwe haveto deal witha probability
scale
runningfrom0 to 100%. As a simplemethodof
we use multipliers
summarizing
that relate to the
distanceto the nearerextreme,
thoughwe couldrephrasethe methodso as to treat50% like the zero
pointoftheCliffscale.
The modifiersformthreeclasses of expressions:
that is
(a) thosewhere"very"reducesa probability
less than 50%, (b) thosewhere"very"inoriginally
that is originally
creasesa probability
greaterthan
50%,and (c) thosewhere"not"(in-,im-,un-)changes
fromoriginally
than50% to less
an expression
greater
than 50%. We findit convenientto workwithdistances fromthe extremesand to relatethe initial
distancefromthenearerextreme(0% or 100%)to the
finaldistancefromits nearerextreme.Whenwe do
summarizes
the
this,a singlemultiplicative
parameter
If x is theinitialvalueand
effect
ofall themodifiers.
then
value,and k is themultiplier,
Xmodis themodified
relationsforthethreeclassesare:
theestimated
(a)
Xvery =
kx
where"very"reducestheestimate),
(expressions
(b)
100
Xvey
k(100 - x)
where"very"increasestheestimate),
(expressions
(c)
Xnot=
k(100- x)
ofnegation:not,in-,un,im-).
(expressions
thesinglevaluek = 1/2,works
Withtheseconventions,
wellforall threegroupsusingtheunweighted
averages
orthesciencewriters'
(orweighted
data) ofall studies
fromTable 1. (We used the averagelog of the ratio
ofthesecondcolumnto thefirstin Tables 4, 5 and 6


TABLE
The effectsofmodifierson the averageprobability

forvarious
expressions.Class (a): "Very"reducesthe estimate.
Class (a)
Infrequent
Lowprobability
Unlikely
Improbable
Seldom
Rarely
Xvery
17
16
17
16
12
9
7
6
11
7
7
4
TABLE
/2x
8.5
8
8.5
8
6
4.5

forvarious
expressions.Class (b): "Very"increasesthe estimate.
Class (b)
Frequent
Highprobability
Likely
Probable
Often
100 - xve,
100-x
45
16
31
30
35
19
9
18
18
15
TABLE
1/2(100- x)
22.5
8
15.5
15
17.5

forvarious
expressions.Class (c): Negationsends the estimatetowardthe
oppositeend of thescale.
Class (c)
Frequent
Probable
Likely
Often
Veryoften
Usually
100- x
xnot
1/2(100- x)
45
30
31
35
17
16
17
15
22.5
15
15.5
17.5
15
23
13
19
7.5a
11.5a
aSubstantial deviation.
to get the 0.5.) A multiplier

of 1/2 forthe changein
distanceto the nearerextremeworkswellfornearly
ofthelasttwocolumns
all expressions,
as comparison
foreach class shows.
In class (c) veryoften(converting
to notveryoften)
to unusually)
havelargeresidand usually(converting
uals forreasonseach readerwillbe able to speculate
about.Outlieranalysisdoes not suggestthemto be
someofus willfeelthatunusunevertheless
extreme,
allyis nota satisfactory
negationoftheprobabilistic
slant.
meaningof usuallybut has an extradifferent
the veryof notveryoftenseemsto soften
Similarly,
the notratherthan to strengthen
the often.At any
rate,whetherthese are includedor not,a value of
k = 1/2is a usefulruleofthumb.
Reyna[14] also studiedthe effectsof negationon
ofthingshappening.
Like ours,manyof
probabilities
hersubjectsdid not assignthe extremeprobabilities
to expressions
For example,59%
implying
certainty.
didnotassign0 to impossible.
Stabilityover Time
Simpson[18] studiedthevariability
ofquantitative
meaningsfor20 qualitativetermsovertimeby comparingresponsesto twoquestionnaires,
onecompleted
in 1942by 335 studentsand theotherin 1962by 395
students.He usedthemidpoint
oftherangegivenby
the studentforeach expressionas its locationvalue.
For the 20 terms,the averageabsolutedeviationbetweenthe meansof the midpointsforthe 1942 and
1962 responseswas only2.0%. Thus the resultsare
similar.
strikingly
Translation
Grigoriu
and Mihaescu[25] translated30 probabilityexpressions
intoRomanianequivalentsand found
"theaveragenumerical
valuesweresimilarfordifferentprofessional
groups[physicians,
medicalstudents,
and medicalrelatedprofessionals]
and veryclose to
the values reportedin the Englishliterature"(page
364).Among16 expressions
thatappearbothin their
studyand amongthe 52 entriesin Table 1, the unweightedaveragesgivenin Table 1 differed
by 10
percentage
pointsor morefor5 expressions.
Contextand Translation
Beyth-Marom
[22] reportsan elaborateset of exin
periments
carriedoutin a forecasting
organization
Israel with 30 expressionsin Hebrew.These were
translated
intoEnglish,and 8 areamongour52. With
respectto context,she foundmorevariationin the
whentheywere
numerical
evaluationsofexpressions
givenin a contextofthe likelihoodof futurespecific
eventsthanwhenmerelygivenas expressions
to be
evaluated(she speaks of presentingexpressionsin
in commonwithourstudy
isolation).The expressions
in isolationcloseto
producednumerical
probabilities
ours,in spiteofthetranslation
problem.
Order
Orderof presentation
is one kindof context,and
ourFormA versusFormB resultsoffersomeinformationon thistopic.Forfivehigh-probability
expressionswithaveragesbetween90 and 100,thegainwas
smallforFormA overFormB, averaging
just 0.3%.
Forthe 17 expressions
withaveragesbetween50 and
90%,thegainforFormA overFormB averaged2.5%.
For8 expressions
withaveragesbetween20 and 50%,
the gainaveraged1.3%. For 22 expressions
withaveragesbetween0 and 20%,theaveragegainwas negative,-0.3%. Thus,forthemostpart,FormA (listing
fromhighto low)led to slightly
higheraveragesthan
FormB (lowto high),exceptforthevery-low-probawhereFormB tendedto produce
bilityexpressions,
slightly
highervalues.We werepleasedthatorderhad

10
not matteredmuch,thoughwe fearedit mightand

thetwoforms.
thatfearmotivated
Scale
SimilarlyKong,Barnett,Mostellerand Youtz [8]

made
to the respondent
foundthatthe scale offered
somedifference
in theaverageestimate.Forexample,
foralmostcertainsubjectswho were offereda free
choice averaged78%; those offeredchoices on an
equallyspacedscale (bytens)averaged82%,notmuch
scaleemphaa spread-out
different;
and thoseoffered
averaged90%, subsizingthe veryhighpercentages
stantially
higher.
DISCUSSION
People oftensay thattheywouldpreferto use the

actualnumbersiftheywereavailableratherthanuse
Those whowriteaboutquanqualitativeexpressions.
in the
titativethingswillusuallyfindthatsomewhere
and commucourseoftheexpositionit is convenient
nicativeto escapefromthenumericalmodeand give
a collectiveidea ratherthan a specificnumber-for
example,to say of a collectionof eventsthat they
oftenhappenorthattheyrarelyhappen.In addition,
thatlay readersacquirebetter
it is notat all verified
information
froma numberthan froma qualitative
expression.That issue is open forresearch.It probably dependson the readerand the messagebeing
delivered.
We nowturnbackto theproposalforcodification.
Someprogress
produced
suggested
bytheinformation
suggests
byotherstudiesand fromthesciencewriters
thatseveralexpressions
amongthe52 notbe usedfor
codification,
especiallythosein the bottompanel of
Table 3: liableto happen(withits overtoneof risk),
notinfrequent
(doublenegative),not unsometimes,
reasonable(logicalratherthan probabilistic),
might
happen,and possible(because of manydistinctbut
definitemeanings).All have largeIQRs and low acmodes.
ceptability
For extremeprobabilities
suchas 98% or 99%, aland
waysand certain,and for2% and 1%, impossible
neverare available.
offersa good spreadwithits
The stemprobability
fromabout 5 to about 90%. We did not
modifiers
includeverywith the stem chance,but using our
k= 1/2,
wecouldimputeabout90% to veryhighchance
and about 7% to verylow chance and the stem
chancethenwouldoffera rangesimilarto thatofthe
stemprobability.
The stemsfrequent,
likely,
probable,
and oftendo not coveras,widea range,thoughthey
each.
offer
nearlyequivalentsetsoffourexpressions
Precisely50% is well naileddownby evenchance
and as oftenas not. A similar value expressing
would be deliveredby moderate
more uncertainty
probability.
Statisticiansmay wish to participatein the next

stageof assessment.Althoughpopularviewsof the
should
probabilities
associatedwiththeseexpressions
help guidefinalchoicesto avoid strainingthe lanof
guage,thedecisionsto choosespecificcodifications
eitherkindmuststillbe onesthatscientistsare comfortablewithand ones thatdo not call forunnecessarilyfineortoo manydistinctions.
APPENDIX
Choice of Expressions
expressionsincludedin thisstudy
The probability
emergedfromsearchesand frompaperssentto us by
scholars.AugustineKong in 1983 made a computer
meanings
searchforpapersdealingwithquantitative
ofprobabilistic
He found18 articles,and
expressions.
theirreferences
led to others.Afterpublicationof
Kong,Barnett,Mostellerand Youtz [8],severalreaders kindlywroteus about additionalpapers. From
about40 articleswe foundmorethan300expressions,
Our
manybeingevaluatedbya sampleofrespondents.
colleagueTimothyReaganreviewedand sortedthem
into three categories:probability,frequency,and
other.AfterreviewingReagan's analysis,Lincoln
forthis
Moses and F. Mostellerchose52 expressions
study.One aim was to choose a set of expressions
whoseassociatednumerical
averageswouldcoverwell
the rangeof probabilities
(expressedin percentages)
from0 to 100%.
Respondents
in thestudiesthatwe summarize
Mostrespondents
andhealthworkinTable 1 werestudents,
physicians,
ers.Newdatainthecurrent
studycomefrommembers
of an associationof sciencewriters.Theirviewsare
inforcommunicate
becausesciencewriters
important
mationfromscientiststo the public,includingother
scientists.
ofthe governing
Withthe cooperation
bodyofthe
ofScienceWriting,
CouncilfortheAdvancement
Inc.,
membersoftheNationalAssociationof
we informed
of
theirnewsletter
ScienceWriters(NASW) through
the proposedsurveyof sciencewriters.Barbara J.
Culliton,Presidentof the Council,wrotea letterto
accompanyour questionnairesent to the NASW
members.
The Questionnaire
to givetheprobabilwereasked,first,
Respondents
ity (as a percentageexpressedto the nearestwhole
numberfrom0 to 100) that theypersonallywould
in the
attachto eachoftheexpressions.
(Respondents
studiesin Table 1 oftenansweredsuch a question.)
Second,theywereaskedto givetherangeofprobabilitiesthattheythoughttheirreaderswouldassociate

The rangegivessomeideaofthe
withthatexpression.
associatedwithan expression;substantial
variability
about meaningcould raise questions
disagreements
Althoughone does
aboutthe value forcodification.
fromsuchan inquiry,
notexpectdelicatedistinctions
to offer25-30 as a rangeas
it seemsverydifferent
opposedto 5-85,f6rinstance.
The questionnairegroupedthe expressionsby
stems,as in the set based on the stem likely:very
and veryunlikely.The stems
likely,likely,unlikely,
happen,likely,
werealways,certain,chance,frequent,
selprobable,rarely,
probability,
possible,
never,often,
meandom,and usually.Based on theirgrammatical
ings,the expressionswereorderedforeach stem.If
the orderhad been haphazard,or the itemsspread
respondents
conscientious
thequestionnaire,
through
wouldhaverequireda longtimeto completethetask,
becauseto be consistenttheywouldhave to huntup
theirestimatesforotherexpressionswiththe same
btem.
In FormA
We usedtwoformsofthequestionnaire.
expressionsforeach stem were orderedfromhigh
to low,and in FormB fromlow to high.
probability
mightmakea difference,
choicesofordering
Arbitrary
and a designwitha balancingapproachusingalso the
againstbias.We
reverseorderoffered
someprotection
and thencombinedthe
analyzedtheformsseparately
results.
The Mailings and Response Rate
to
On May 6, 1987,we mailed637 questionnaires
membersof the NASW in the United States and
mailingto the
Canada.On June4 we senta follow-up
Fromthetwomailings
475whohadnotyetresponded.
withmainlyusableresponses;
we received238 replies2
orreturned
undelivered
about5% wereeitherreturned
blankbythesciencewriters.
Editingthe Responses
recontrolled
conditions,
Underthemostcarefully
and
spondentsdo as theyplease, as experimenters
surveyscientistswell know.Althoughwe asked the
some
to giveestimatesin wholenumbers,
respondents
gaveanswerssuchas 0.001,0.02,<50, >70, 1+, 60-,
10-20, whichrequiredediting.Some changeswere
simple.For example,we changed0.0001to 0 and 1020 to the midpoint,15. Whenwe could not make a
we changedtheresponseto a
reasonableadjustment,
blank.
itis a familiar
In workwithscalesinquestionnaires,
findingthat respondentsoccasionallyget turned
ofthe numberthey
aroundand givethe complement
intend.For example,in ourstudyalwaysmighthave
produceda personalestimateof 2% and a rangeof
0-5%, whereasthe respondentintended98% and
95-100%. When such errorsoccurredon the more
11
extremeexpressions,we changedthe estimatesto

theircomplements.
ACKNOWLEDGMENTS
We thankConstanceBrown,BarbaraCulliton,John
D. Emerson,David C. Hoaglin,Lincoln E. Moses,
Marjorie Olson, TimothyR. Reagan, Christopher
Schmid,KatherineTaylor-Halvorsen
and Wallace
Waterfallfortheirmanythoughtful
commentsand
Thisworkwasfacilitated
suggestions.
inpartbyGrant
SES-84-01422fromtheNationalScienceFoundation.
REFERENCES AND NOTES
of
[1] BRYANT, G. D. and NORMAN, G. R. (1980). Expressions
New EnglandJ. Med. 302

Wordsand numbers.
probability:
411.
twiceeach; averagesread
n = 32: 16 physiciansresponding
fromchart.
in
[2] BUDESCU,D. V. and WALLSTEN, T. S. (1985).Consistency
Behavphrases.Organizational
ofprobabilistic
interpretation
iorandHumanDecisionProcesses36 391-405.
University
inpsychology,
students
n = 32:faculty
andgraduate
ofNorthCarolina,ChapelHill.
[3] HAKEL, M. D. (1968).Howoftenis often?Amer.PsychoL23
533-534.
course,Unipsychology
n = 100:studentsin an introductory
versity
ofMinnesota.
[4] HARTLEY, J., TRUEMAN, M. and RODGERS, A. (1984). The
on questionnaire
effects
of verbaland numericalquantifiers
15.2 149-155.
responses.
Appl.Ergonomics
ofKeele:na = 20 students
University
students,
Undergraduate
studentsgiven
givenone set ofexpressions.
nb = 20 different
givenstillanotherset.a: always,
anotherset.nc= 20 students
fairlymany
seldom,never.b: always,often,
often,
occasionalIy,
never.c: always,quiteoften,sometimes,
times,occasionally,
noneofthetime.
infrequently,
of subjective
proba[51 HARTSOUGH, W. R. (1977).Assignment
oflocusof
phrasesas a function
bilitiesto verbalprobability
95 87-97.
J.Psychology
controland setconditions.
n = 60 (10 each in 6 groups):studentsin an introductory
course.
psychology
[6] JOHNSON, E. M. (1973). Numericalencodingof qualitative
of uncertainty.
TechnicalPaper 250,U.S. Army
expressions
fortheBehavioraland SocialSciences,AD
ResearchInstitute
780814.
n = 28: 14 U.S. Armyenlistedmenand 14 extensioncollege
students.
[7] KENNEY, R. M. (1981).Betweenneverand always.NewEnglandJ. Med.305 1097-1098.
Massachusetts
n = 24: membersof PathologyDepartment,
GeneralHospital.
[8] KONG, A., BARNETT, G. O., MOSTELLER, F. and YOUTZ, C.
evaluateexpressionsof
(1986). How medicalprofessionals
NewEnglandJ. Med.315 740-744.
probability.
scale.n0
scale.nb- 134.Uniform
na ; 140.Highprobability
- 170.Freechoice.nd 144.Lowprobability
scale.Physicians,
and auxiliarystudentsand nurses,namedical,non-medical,
tionwide.
[9] LICHTENSTEIN, S. andNEWMAN, J.R. (1967).Empiricalscalprobverbalphrasesassociatedwithnumerical
ingofcommon
Sciences9 563-564.
abilities.Psychonomic
employees.
n = 186:SystemDevelopment
Corporation

12
[10] LEVINE,J.M. and ELDREDGE,D. (December 1970). The effects
performance.
uponphotointerpreter
ofancillaryinformation
AmericanInstitutesforResearch,InstituteforResearchin
AIR-20131-12/70-FR.
Office,
Washington
Psychobiology,
n = 20:enlistedU.S. Armyimageinterpreters.
[11] MAPES, R. E. A. (1979). Verbaland numericalestimatesof
contexts.SocialScienceand Mediin therapeutic
probability
cine13A 277-282.
withchlor"Sideeffects
givenexpression
na = 29: physicians
givenexpression
arefrequent."
amphenicol
nb = 33:physicians
withneomycin
sulphatearefrequent."
"Sideeffects
[12] NAKAO, M. A. and AXELROD, S. (1983).Numbersare better
of frequency
haveno place
thanwords:Verbalspecifications
Amer.J. Med.74 1061-1065.
in medicine.
Means read from
nb =106 physicians.
na = 103 physicians.
chart.
[13] REAGAN, R. T., MOSTELLER, F. and YOUTZ, C. (1989).The
quantitativemeaningsof verbal probabilityexpressions.
74 433-442.
J.Appl.Psychology
in a psychology
course,Stanford
n = 115: undergraduates
University.
and proba[14] REYNA, V. F. (1981).The languageofpossibility
and Cognition
Memory
ofnegationon meaning.
bility:Effects
9 642-650.
n = 41 adultvolunteers.
[15] ROBERTS, D. E. and GUPTA, G. (1987). To the editor.New
EnglandJ. Med. 316 550.
na = 45 house staff.nb = 24 attendingphysicians.
ofwords.
themeanings
[16] ROBERTSON, W. 0. (1983).Quantifying
J.Amer.Med.Assoc.249 2631-2632.
at the
na = 53: Seattlephysicians.
nb= 80: graduatestudents
School of BusinessAdministraof Washington's
University
Orthopedic
tion.n0= 40: BoardofTrusteesat theChildren's
Hospitaland MedicalCenter,Seattle.
to rareevents.
probabilities
[17] SELVIDGE, J. (1972). Assigning
GraduateSchoolofBusinessAdministraPh.D. dissertation,
tion, George F. Baker Foundation, Harvard Univ.
Subjectswere HarvardBusinessSchool studentsin MBA
program.na = 59: Estimates made on basis of a statement
without
context.nb=127: Contextswereprovided.
Also in Mosteller,F. (1977). Assessingunknownnumbers:
andPublicPolicy
In Statistics
OrderofmagnitudTe
estimation.
(W. B. Fairleyand F. Mosteller,eds.) 163-184.AddisonWesley,Reading,Mass.
in meanings
forquantitative
[18] SIMPSON, R. H. (1963).Stability
over20 years.Q. J. Speech49 146-151.
terms:A comparison
1942study.na = 335:86 highschooland 249 collegestudents.

students.
1962study.nb= 395university
[19] TOoGOOD, J. H. (1980). What do we mean by "usually"?
Lancet 1 1094.
secondary
technologists,
nurses,laboratory
n = 51:physicians,
and engineers.
schoolteachers,
fromsciencewriters.
estimates
study,
[20] Current
to
fromexpression
Variessomewhat
n = 230:sciencewriters.
211-237.
expression,
[21] BARTHOLOMEW, D. J. (1961).A methodofallowingfor"notAppl.Statist.10 52-59.
at-home"
bias in samplesurveys.
[22] BEYTH-MAROM, R. (1982).How probableis probable?A nuJ. Foreexpressions.
ofverbalprobability
mericaltranslation
casting1 257-269.
ofa decisionhowthenation
[23] BOFFEY, P. M. (1976).Anatomy
declaredwar on swine flu.Science 192 636-641.
Psychol.Rev. 66 27as multipliers.
[24] CLIFF, N. (1959).Adverbs
on quantisymposium
a four-paper
44. Thispaperstimulated
ofadverbson themeaningofadjectives:
fication
oftheeffect
Chance1 (3) 32-51(1988).
[25] GRIGORIU, B. D. andMIHAESCU, T. (1988).Evaluareanumerfolositein limbajulmedical.
de probabilitate
ica a expresiilor
Rev. Med. Chir. Soc. Med. Nat. Iasi 92 361-364.
[26] HANSEN, M. H. and HURWITZ, W. N. (1946).The problemof

J. Amer.Statist.Assoc.41
in samplesurveys.
non-response
517-529.
[27] KAHNEMAN, D. SLOVIC, P. and TVERSKY, A., eds. (1982).
Judgmentunder Uncertainty:Heuristics and Biases. Cam-
bridgeUniv.Press,Cambridge.
the "possibil[28] MOSTELLER, F. (1976).Swineflu:Quantifying
ity."Science 192 1286, 1288.
errors.In Interna[29] MOSTELLER, F. (1978). I. Non-sampling
tional Encyclopedia of Statistics (W. H. Kruskal and J. M.
Tanur,eds.) 1 208-229.The FreePress,NewYork.

frequently
[30] PEPPER, S. andPRYTULAK, L. S. (1974).Sometimes
ofquanin theinterpretation
meansseldom:Contexteffects
titativeexpressions.J. Res. in Personality8 95-101.

[31] SLOVIC, P., FISCHOFF, B. andLICHTENSTEIN, S. (1982).Facts
under
risk.In Judgment
perceived
versusfears:Understanding
Uncertainty:Heuristics and Biases (D. Kahneman, P. Slovic
and A. Tversky,eds.) 463-489. CambridgeUniv. Press,

Cambridge.
[32] TVERSKY, A. and KAHNEMAN, D. (1981). The framingof
ofchoice.Science2 1 1 453-458.
decisionsandthepsychology
Univ.Press,
[33] KENT, S. (1949).StrategicIntelligence.Princeton
N.J.
Princeton,
Comment
HerbertH. Clark
In the last fewyears,Mosteller,Youtz and their
and frequency
colleagueshave lookedat probability
freimprobable,
suchas usual,verylikely,
expressions
quentand as oftenas not.Their interestis in how
technicalinthesetermsare used in communicating
Stanford
ofPsychology,
HerbertH. Clarkis Professor
94305.
California
Stanford,
University,
and theirgoal is to betterthatcommuniformation,

cation,to makeit moreprecise.Theirprojecthas two
whatthese
phases. In the first,theywill determine
In
termsmeanto thepeoplewhouse them. theirown
studytheyhave found,forexample,thatfrequentis
of about
judgedto representan averageproportion
of about
range
interquartile
an
0.72 ofthetimewith
you
claim,
they
is
frequent,
If
something
0.15. yousay
plus
time
72%
of
the
occurs
about
it
that
are saying


Quantifying Probabilistic Expressions

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Quantifying Probabilistic Expressions

Uploaded by

Copyright:

Available Formats

Quantifying Probabilistic Expressions

Author(s): Frederick Mosteller and Cleo Youtz

This content downloaded from 171.67.34.205 on Fri, 27 Dec 2013 20:49:43 PM

This content downloaded from 171.67.34.205 on Fri, 27 Dec 2013 20:49:43 PM