9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

Designofexperiments
FromWikipedia,thefreeencyclopedia

Forthebook,seeTheDesignofExperiments.
Ingeneralusage,designofexperiments(DOE)or
experimentaldesignisthedesignofanyinformationgathering
exerciseswherevariationispresent,whetherunderthefull
controloftheexperimenterornot.However,instatistics,these
termsareusuallyusedforcontrolledexperiments.Formal
plannedexperimentationisoftenusedinevaluatingphysical
objects,chemicalformulations,structures,components,and
materials.Othertypesofstudy,andtheirdesign,arediscussedin
thearticlesoncomputerexperiments,opinionpollsandstatistical
surveys(whicharetypesofobservationalstudy),natural
experimentsandquasiexperiments(forexample,quasi
experimentaldesign).SeeExperimentforthedistinctionbetween
thesetypesofexperimentsorstudies.

Designofexperimentswithfull
factorialdesign(left),response
surfacewithseconddegree
polynomial(right)

Inthedesignofexperiments,theexperimenterisofteninterestedintheeffectofsomeprocessor
intervention(the"treatment")onsomeobjects(the"experimentalunits"),whichmaybepeople,partsof
people,groupsofpeople,plants,animals,etc.Designofexperimentsisthusadisciplinethathasvery
broadapplicationacrossallthenaturalandsocialsciencesandengineering.

Contents
1History
1.1Systematicclinicaltrials
1.2Statisticalexperiments,followingCharlesS.Peirce
1.2.1Randomizedexperiments
1.2.2Optimaldesignsforregressionmodels
1.3Sequencesofexperiments
2Fisher'sprinciples
3Example
4Avoidingfalsepositives
5Discussiontopicswhensettingupanexperimentaldesign
6Statisticalcontrol
7ExperimentaldesignsafterFisher
8Humanparticipantexperimentaldesignconstraints
9Seealso
10Notes
11References
12Furtherreading
13Externallinks

History
Systematicclinicaltrials
https://en.wikipedia.org/wiki/Design_of_experiments

1/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

In1747,whileservingassurgeononHMSSalisbury,JamesLindcarriedoutasystematicclinicaltrial
tocompareremediesforscurvy.[1]
Lindselected12menfromtheship,allsufferingfromscurvy.Lindlimitedhissubjectstomenwho
"wereassimilarasIcouldhavethem",thatisheprovidedstrictentryrequirementstoreduceextraneous
variation.Hedividedthemintosixpairs,givingeachpairdifferentsupplementstotheirbasicdietfor
twoweeks.Thetreatmentswereallremediesthathadbeenproposed:
Aquartofcidereveryday.
Twentyfivegutts(drops)ofvitriol(sulphuricacid)threetimesadayuponanemptystomach.
Onehalfpintofseawatereveryday.
Amixtureofgarlic,mustard,andhorseradishinalumpthesizeofanutmeg.
Twospoonfulsofvinegarthreetimesaday.
Twoorangesandonelemoneveryday.
Thecitrustreatmentstoppedaftersixdayswhentheyranoutoffruit,butbythattimeonesailorwasfit
fordutywhiletheotherhadalmostrecovered.Apartfromthat,onlygroupone(cider)showedsome
effectofitstreatment.Theremainderofthecrewpresumablyservedasacontrol,butLinddidnotreport
resultsfromanycontrol(untreated)group.

Statisticalexperiments,followingCharlesS.Peirce
Mainarticle:Frequentiststatistics
Seealso:Randomization
AtheoryofstatisticalinferencewasdevelopedbyCharlesS.Peircein"IllustrationsoftheLogicof
Science"(18771878)and"ATheoryofProbableInference"(1883),twopublicationsthatemphasized
theimportanceofrandomizationbasedinferenceinstatistics.
Randomizedexperiments
Mainarticle:Randomassignment
Seealso:Repeatedmeasuresdesign
CharlesS.Peircerandomlyassignedvolunteerstoablinded,repeatedmeasuresdesigntoevaluatetheir
abilitytodiscriminateweights.[2][3][4][5]Peirce'sexperimentinspiredotherresearchersinpsychologyand
education,whichdevelopedaresearchtraditionofrandomizedexperimentsinlaboratoriesand
specializedtextbooksinthe1800s.[2][3][4][5]
Optimaldesignsforregressionmodels
Mainarticle:Responsesurfacemethodology
Seealso:Optimaldesign
CharlesS.PeircealsocontributedthefirstEnglishlanguagepublicationonanoptimaldesignfor
regressionmodelsin1876.[6]Apioneeringoptimaldesignforpolynomialregressionwassuggestedby
Gergonnein1815.In1918KirstineSmithpublishedoptimaldesignsforpolynomialsofdegreesix(and
less).

Sequencesofexperiments
https://en.wikipedia.org/wiki/Design_of_experiments

2/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

Mainarticle:Sequentialanalysis
Seealso:Multiarmedbanditproblem,GittinsindexandOptimaldesign
Theuseofasequenceofexperiments,wherethedesignofeachmaydependontheresultsofprevious
experiments,includingthepossibledecisiontostopexperimenting,iswithinthescopeofSequential
analysis,afieldthatwaspioneered[7]byAbrahamWaldinthecontextofsequentialtestsofstatistical
hypotheses.[8]HermanChernoffwroteanoverviewofoptimalsequentialdesigns,[9]whileadaptive
designshavebeensurveyedbyS.Zacks.[10]Onespecifictypeofsequentialdesignisthe"twoarmed
bandit",generalizedtothemultiarmedbandit,onwhichearlyworkwasdonebyHerbertRobbinsin
1952.[11]

Fisher'sprinciples
AmethodologyfordesigningexperimentswasproposedbyRonald
Fisher,inhisinnovativebooks:TheArrangementofField
Experiments(1926)andTheDesignofExperiments(1935).Muchof
hispioneeringworkdealtwithagriculturalapplicationsofstatistical
methods.Asamundaneexample,hedescribedhowtotestthelady
tastingteahypothesis,thatacertainladycoulddistinguishby
flavouralonewhetherthemilkortheteawasfirstplacedinthecup.
Thesemethodshavebeenbroadlyadaptedinthephysicalandsocial
sciences,arestillusedinagriculturalengineeringanddifferfromthe
designandanalysisofcomputerexperiments.
Comparison
Insomefieldsofstudyitisnotpossibletohaveindependent
measurementstoatraceablemetrologystandard.Comparisons
betweentreatmentsaremuchmorevaluableandareusually
preferable,andoftencomparedagainstascientificcontrolor
traditionaltreatmentthatactsasbaseline.

RonaldFisher

Randomization
Randomassignmentistheprocessofassigningindividualsatrandomtogroupsortodifferent
groupsinanexperiment.Therandomassignmentofindividualstogroups(orconditionswithina
group)distinguishesarigorous,"true"experimentfromanobservationalstudyor"quasi
experiment".[12]Thereisanextensivebodyofmathematicaltheorythatexplorestheconsequences
ofmakingtheallocationofunitstotreatmentsbymeansofsomerandommechanismsuchas
tablesofrandomnumbers,ortheuseofrandomizationdevicessuchasplayingcardsordice.
Assigningunitstotreatmentsatrandomtendstomitigateconfounding,whichmakeseffectsdue
tofactorsotherthanthetreatmenttoappeartoresultfromthetreatment.Therisksassociatedwith
randomallocation(suchashavingaseriousimbalanceinakeycharacteristicbetweenatreatment
groupandacontrolgroup)arecalculableandhencecanbemanageddowntoanacceptablelevel
byusingenoughexperimentalunits.Theresultsofanexperimentcanbegeneralizedreliablyfrom
theexperimentalunitstoalargerstatisticalpopulationofunitsonlyiftheexperimentalunitsarea
randomsamplefromthelargerpopulationtheprobableerrorofsuchanextrapolationdependson
thesamplesize,amongotherthings.
Statisticalreplication
Measurementsareusuallysubjecttovariationandmeasurementuncertaintythustheyare
repeatedandfullexperimentsarereplicatedtohelpidentifythesourcesofvariation,tobetter
estimatethetrueeffectsoftreatments,tofurtherstrengthentheexperiment'sreliabilityand
validity,andtoaddtotheexistingknowledgeofthetopic.[13]However,certainconditionsmustbe
https://en.wikipedia.org/wiki/Design_of_experiments

3/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

metbeforethereplicationoftheexperimentiscommenced:theoriginalresearchquestionhas
beenpublishedinapeerreviewedjournalorwidelycited,theresearcherisindependentofthe
originalexperiment,theresearchermustfirsttrytoreplicatetheoriginalfindingsusingthe
originaldata,andthewriteupshouldstatethatthestudyconductedisareplicationstudythattried
tofollowtheoriginalstudyasstrictlyaspossible.[14]
Blocking
Blockingisthearrangementofexperimentalunitsintogroups(blocks/lots)consistingofunitsthat
aresimilartooneanother.Blockingreducesknownbutirrelevantsourcesofvariationbetween
unitsandthusallowsgreaterprecisionintheestimationofthesourceofvariationunderstudy.
Orthogonality
Orthogonalityconcernstheformsofcomparison
(contrasts)thatcanbelegitimatelyandefficientlycarried
out.Contrastscanberepresentedbyvectorsandsetsof
orthogonalcontrastsareuncorrelatedandindependently
distributedifthedataarenormal.Becauseofthis
independence,eachorthogonaltreatmentprovides
differentinformationtotheothers.IfthereareTtreatments
andT1orthogonalcontrasts,alltheinformationthatcan
becapturedfromtheexperimentisobtainablefromtheset
ofcontrasts.

Exampleoforthogonalfactorial
design

Factorialexperiments
Useoffactorialexperimentsinsteadoftheonefactoratatimemethod.Theseareefficientat
evaluatingtheeffectsandpossibleinteractionsofseveralfactors(independentvariables).Analysis
ofexperimentdesignisbuiltonthefoundationoftheanalysisofvariance,acollectionofmodels
thatpartitiontheobservedvarianceintocomponents,accordingtowhatfactorstheexperiment
mustestimateortest.

Example
ThisexampleisattributedtoHaroldHotelling.[9]Itconveys
someoftheflavorofthoseaspectsofthesubjectthatinvolve
combinatorialdesigns.
Weightsofeightobjectsaremeasuredusingapanbalanceand
setofstandardweights.Eachweighingmeasurestheweight
differencebetweenobjectsintheleftpanvs.anyobjectsinthe
rightpanbyaddingcalibratedweightstothelighterpanuntil
thebalanceisinequilibrium.Eachmeasurementhasarandom
error.Theaverageerroriszerothestandarddeviationsofthe
probabilitydistributionoftheerrorsisthesamenumberon
differentweighingsanderrorsondifferentweighingsare
independent.Denotethetrueweightsby

Weconsidertwodifferentexperiments:
1. Weigheachobjectinonepan,withtheotherpanempty.LetXibethemeasuredweightoftheith
object,fori=1,...,8.
https://en.wikipedia.org/wiki/Design_of_experiments

4/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

2. DotheeightweighingsaccordingtothefollowingscheduleandletYibethemeasureddifference
fori=1,...,8:

Thentheestimatedvalueoftheweight1is

Similarestimatescanbefoundfortheweightsoftheotheritems.Forexample

Thequestionofdesignofexperimentsis:whichexperimentisbetter?
ThevarianceoftheestimateX1of1is2ifweusethefirstexperiment.Butifweusethesecond
experiment,thevarianceoftheestimategivenaboveis2/8.Thusthesecondexperimentgivesus8
timesasmuchprecisionfortheestimateofasingleitem,andestimatesallitemssimultaneously,with
thesameprecision.Whatthesecondexperimentachieveswitheightwouldrequire64weighingsifthe
itemsareweighedseparately.However,notethattheestimatesfortheitemsobtainedinthesecond
experimenthaveerrorsthatcorrelatewitheachother.
Manyproblemsofthedesignofexperimentsinvolvecombinatorialdesigns,asinthisexampleand
others.[15]

Avoidingfalsepositives
Falsepositiveconclusions,oftenresultingfromthepressuretopublishortheauthor'sownconfirmation
bias,areaninherenthazardinmanyfields,andexperimentaldesignswithundiscloseddegreesof
freedomareaproblem.[16]Thiscanleadtoconsciousorunconscious"phacking":tryingmultiplethings
untilyougetthedesiredresult.Ittypicallyinvolvesthemanipulationperhapsunconsciouslyofthe
processofstatisticalanalysisandthedegreesoffreedomuntiltheyreturnafigurebelowthep<.05level
ofstatisticalsignificance.[17][18]Sothedesignoftheexperimentshouldincludeaclearstatement
proposingtheanalysestobeundertaken.
Clearandcompletedocumentationoftheexperimentalmethodologyisalsoimportantinorderto
supportreplicationofresults.[19]
https://en.wikipedia.org/wiki/Design_of_experiments

5/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

Discussiontopicswhensettingupanexperimentaldesign
Anexperimentaldesignorrandomizedclinicaltrialrequirescarefulconsiderationofseveralfactors
beforeactuallydoingtheexperiment.[20]Anexperimentaldesignisthelayingoutofadetailed
experimentalplaninadvanceofdoingtheexperiment.Someofthefollowingtopicshavealreadybeen
discussedintheprinciplesofexperimentaldesignsection:
1. Howmanyfactorsdoesthedesignhave?andarethelevelsofthesefactorsfixedorrandom?
2. Arecontrolconditionsneeded,andwhatshouldtheybe?
3. Manipulationchecksdidthemanipulationreallywork?
4. Whatarethebackgroundvariables?
5. Whatisthesamplesize.Howmanyunitsmustbecollectedfortheexperimenttobegeneralisable
andhaveenoughpower?
6. Whatistherelevanceofinteractionsbetweenfactors?
7. Whatistheinfluenceofdelayedeffectsofsubstantivefactorsonoutcomes?
8. Howdoresponseshiftsaffectselfreportmeasures?
9. Howfeasibleisrepeatedadministrationofthesamemeasurementinstrumentstothesameunitsat
differentoccasions,withaposttestandfollowuptests?
10. Whataboutusingaproxypretest?
11. Aretherelurkingvariables?
12. Shouldtheclient/patient,researcheroreventheanalystofthedatabeblindtoconditions?
13. Whatisthefeasibilityofsubsequentapplicationofdifferentconditionstothesameunits?
14. Howmanyofeachcontrolandnoisefactorsshouldbetakenintoaccount?

Statisticalcontrol
Itisbestthataprocessbeinreasonablestatisticalcontrolpriortoconductingdesignedexperiments.
Whenthisisnotpossible,properblocking,replication,andrandomizationallowforthecarefulconduct
ofdesignedexperiments.[21]Tocontrolfornuisancevariables,researchersinstitutecontrolchecksas
additionalmeasures.Investigatorsshouldensurethatuncontrolledinfluences(e.g.,sourcecredibility
perception)donotskewthefindingsofthestudy.Amanipulationcheckisoneexampleofacontrol
check.Manipulationchecksallowinvestigatorstoisolatethechiefvariablestostrengthensupportthat
thesevariablesareoperatingasplanned.
Oneofthemostimportantrequirementsofexperimentalresearchdesignsisthenecessityofeliminating
theeffectsofspurious,intervening,andantecedentvariables.Inthemostbasicmodel,cause(X)leadsto
effect(Y).Buttherecouldbeathirdvariable(Z)thatinfluences(Y),andXmightnotbethetruecause
atall.Zissaidtobeaspuriousvariableandmustbecontrolledfor.Thesameistrueforintervening
variables(avariableinbetweenthesupposedcause(X)andtheeffect(Y)),andantecedingvariables(a
variablepriortothesupposedcause(X)thatisthetruecause).Whenathirdvariableisinvolvedandhas
notbeencontrolledfor,therelationissaidtobeazeroorderrelationship.Inmostpracticalapplications
ofexperimentalresearchdesignsthereareseveralcauses(X1,X2,X3).Inmostdesigns,onlyoneof
thesecausesismanipulatedatatime.

ExperimentaldesignsafterFisher
Someefficientdesignsforestimatingseveralmaineffectswerefoundindependentlyandinnear
successionbyRajChandraBoseandK.Kishenin1940attheIndianStatisticalInstitute,butremained
littleknownuntilthePlackettBurmandesignswerepublishedinBiometrikain1946.Aboutthesame
time,C.R.Raointroducedtheconceptsoforthogonalarraysasexperimentaldesigns.Thisconcept
https://en.wikipedia.org/wiki/Design_of_experiments

6/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

playedacentralroleinthedevelopmentofTaguchimethodsbyGenichiTaguchi,whichtookplace
duringhisvisittoIndianStatisticalInstituteinearly1950s.Hismethodsweresuccessfullyappliedand
adoptedbyJapaneseandIndianindustriesandsubsequentlywerealsoembracedbyUSindustryalbeit
withsomereservations.
In1950,GertrudeMaryCoxandWilliamGemmellCochranpublishedthebookExperimentalDesigns,
whichbecamethemajorreferenceworkonthedesignofexperimentsforstatisticiansforyears
afterwards.
Developmentsofthetheoryoflinearmodelshaveencompassedandsurpassedthecasesthatconcerned
earlywriters.Today,thetheoryrestsonadvancedtopicsinlinearalgebra,algebraandcombinatorics.
Aswithotherbranchesofstatistics,experimentaldesignispursuedusingbothfrequentistandBayesian
approaches:Inevaluatingstatisticalprocedureslikeexperimentaldesigns,frequentiststatisticsstudies
thesamplingdistributionwhileBayesianstatisticsupdatesaprobabilitydistributionontheparameter
space.
SomeimportantcontributorstothefieldofexperimentaldesignsareC.S.Peirce,R.A.Fisher,F.Yates,
C.R.Rao,R.C.Bose,J.N.Srivastava,ShrikhandeS.S.,D.Raghavarao,W.G.Cochran,O.
Kempthorne,W.T.Federer,V.V.Fedorov,A.S.Hedayat,J.A.Nelder,R.A.Bailey,J.Kiefer,W.J.
Studden,A.Pzman,F.Pukelsheim,D.R.Cox,H.P.Wynn,A.C.Atkinson,G.E.P.BoxandG.
Taguchi.ThetextbooksofD.MontgomeryandR.Myershavereachedgenerationsofstudentsand
practitioners.[22][23][24]

Humanparticipantexperimentaldesignconstraints
Lawsandethicalconsiderationsprecludesomecarefullydesignedexperimentswithhumansubjects.
Legalconstraintsaredependentonjurisdiction.Constraintsmayinvolveinstitutionalreviewboards,
informedconsentandconfidentialityaffectingbothclinical(medical)trialsandbehavioralandsocial
scienceexperiments.[25]Inthefieldoftoxicology,forexample,experimentationisperformedon
laboratoryanimalswiththegoalofdefiningsafeexposurelimitsforhumans.[26]Balancingthe
constraintsareviewsfromthemedicalfield.[27]Regardingtherandomizationofpatients,"...ifnoone
knowswhichtherapyisbetter,thereisnoethicalimperativetouseonetherapyoranother."(p380)
Regardingexperimentaldesign,"...itisclearlynotethicaltoplacesubjectsatrisktocollectdataina
poorlydesignedstudywhenthissituationcanbeeasilyavoided...".(p393)

Seealso
Adversarialcollaboration
Bayesianexperimentaldesign
Clinicaltrial
Computerexperiment
Controlvariable
Controllingforavariable
Experimetrics(econometricsrelatedexperiments)
Factoranalysis
Firstinmanstudy
Glossaryofexperimentaldesign
Greyboxmodel
Instrumenteffect
Lawoflargenumbers
https://en.wikipedia.org/wiki/Design_of_experiments

7/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

Manipulationchecks
Multifactordesignofexperimentssoftware
Probabilisticdesign
Protocol(naturalsciences)
Quasiexperimentaldesign
Randomizedblockdesign
Randomizedcontrolledtrial
Researchdesign
Robustparameterdesign
Supersaturateddesign
Surveysampling
Systemidentification
Taguchimethods

Notes
1. Dunn,Peter(January1997)."JamesLind(171694)ofEdinburghandthetreatmentofscurvy"
(http://fn.bmj.com/cgi/content/full/76/1/F64).ArchiveofDiseaseinChildhoodFoetalNeonatal(United
Kingdom:BritishMedicalJournalPublishingGroup)76(1):6465.doi:10.1136/fn.76.1.F64
(https://dx.doi.org/10.1136%2Ffn.76.1.F64).PMC1720613
(https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1720613).PMID9059193
(https://www.ncbi.nlm.nih.gov/pubmed/9059193).Retrieved20090117.
2. Peirce,CharlesSandersJastrow,Joseph(1885)."OnSmallDifferencesinSensation"
(http://psychclassics.yorku.ca/Peirce/smalldiffs.htm).MemoirsoftheNationalAcademyofSciences3:73
83.
3. Hacking,Ian(September1988)."Telepathy:OriginsofRandomizationinExperimentalDesign".Isis79(3):
427451.doi:10.1086/354775(https://dx.doi.org/10.1086%2F354775).JSTOR234674
(https://www.jstor.org/stable/234674).MR1013489(https://www.ams.org/mathscinetgetitem?mr=1013489).
4. StephenM.Stigler(November1992)."AHistoricalViewofStatisticalConceptsinPsychologyand
EducationalResearch".AmericanJournalofEducation101(1):6070.doi:10.1086/444032
(https://dx.doi.org/10.1086%2F444032).JSTOR1085417(https://www.jstor.org/stable/1085417).
5. TrudyDehue(December1997)."Deception,Efficiency,andRandomGroups:PsychologyandtheGradual
OriginationoftheRandomGroupDesign".Isis88(4):653673.doi:10.1086/383850
(https://dx.doi.org/10.1086%2F383850).PMID9519574(https://www.ncbi.nlm.nih.gov/pubmed/9519574).
6. Peirce,C.S.(1876)."NoteontheTheoryoftheEconomyofResearch".CoastSurveyReport:197201.,
actuallypublished1879,NOAAPDFEprint(http://docs.lib.noaa.gov/rescue/cgs/001_pdf/CSC
0025.PDF#page=222).
ReprintedinCollectedPapers7,paragraphs139157,alsoinWritings4,pp.7278,andinPeirce,C.S.
(JulyAugust1967)."NoteontheTheoryoftheEconomyofResearch".OperationsResearch15(4):643
648.doi:10.1287/opre.15.4.643(https://dx.doi.org/10.1287%2Fopre.15.4.643).JSTOR168276
(https://www.jstor.org/stable/168276).
7. Johnson,N.L.(1961)."Sequentialanalysis:asurvey."JournaloftheRoyalStatisticalSociety,SeriesA.Vol.
124(3),372411.(pages375376)
8. Wald,A.(1945)"SequentialTestsofStatisticalHypotheses",AnnalsofMathematicalStatistics,16(2),117
186.
9. HermanChernoff,SequentialAnalysisandOptimalDesign,SIAMMonograph,1972.
10. Zacks,S.(1996)"AdaptiveDesignsforParametricModels".In:Ghosh,S.andRao,C.R.,(Eds)(1996).
"DesignandAnalysisofExperiments,"HandbookofStatistics,Volume13.NorthHolland.ISBN0444
820612.(pages151180)
11. Robbins,H.(1952)."SomeAspectsoftheSequentialDesignofExperiments".BulletinoftheAmerican
MathematicalSociety58(5):527535.doi:10.1090/S000299041952096208
(https://dx.doi.org/10.1090%2FS000299041952096208).
12. Creswell,J.W.(2008).Educationalresearch:Planning,conducting,andevaluatingquantitativeandqualitative
research(3rd).UpperSaddleRiver,NJ:PrenticeHall.2008,p.300.ISBN0136135501

https://en.wikipedia.org/wiki/Design_of_experiments

8/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

13. Dr.Hani(2009)."Replicationstudy"(http://www.experimentresources.com/replicationstudy.html).
Retrieved27October2011.
14. Burman,LeonardE.RobertW.ReedJamesAlm(2010)."Acallforreplicationstudies"
(http://pfr.sagepub.com)(journalarticle).PublicFinanceReview.pp.787793.doi:10.1177/1091142110385210
(https://dx.doi.org/10.1177%2F1091142110385210).Retrieved27October2011.
15. JackSifri(8December2014)."HowtoUseDesignofExperimentstoCreateRobustDesignsWithHigh
Yield"(https://www.youtube.com/watch?v=hfdZabCVwzc).youtube.com.Retrieved20150211.
16. Simmons,JosephLeifNelsonUriSimonsohn(November2011)."FalsePositivePsychology:Undisclosed
FlexibilityinDataCollectionandAnalysisAllowsPresentingAnythingasSignificant"
(http://pss.sagepub.com/content/22/11/1359.full).PsychologicalScience(WashingtonDC:Associationfor
PsychologicalScience)22(11):13591366.doi:10.1177/0956797611417632
(https://dx.doi.org/10.1177%2F0956797611417632).ISSN09567976(https://www.worldcat.org/issn/0956
7976).PMID22006061(https://www.ncbi.nlm.nih.gov/pubmed/22006061).Retrieved29January2012.
17. "Science,TrustAndPsychologyInCrisis"(http://www.kplu.org/post/sciencetrustandpsychologycrisis).
KPLU.20140602.Retrieved20140612.
18. "WhyStatisticallySignificantStudiesCanBeInsignificant"(http://www.psmag.com/navigation/natureand
technology/statisticallysignificantstudiesarentnecessarilysignificant82832/).PacificStandard.201406
04.Retrieved20140612.
19. ChrisChambers(20140610)."Physicsenvy:Dohardsciencesholdthesolutiontothereplicationcrisisin
psychology?"(http://www.theguardian.com/science/headquarters/2014/jun/10/physicsenvydohardsciences
holdthesolutiontothereplicationcrisisinpsychology).theguardian.com.Retrieved20140612.
20. Ader,Mellenberg&Hand(2008)"AdvisingonResearchMethods:Aconsultant'scompanion"
21. Bisgaard,S(2008)"MustaProcessbeinStatisticalControlbeforeConductingDesignedExperiments?",
QualityEngineering,ASQ,20(2),pp143176
22. Montgomery,Douglas(2013).Designandanalysisofexperiments(8thed.).Hoboken,NJ:JohnWiley&
Sons,Inc.ISBN9781118146927.
23. Walpole,RonaldE.Myers,RaymondH.Myers,SharonL.Ye,Keying(2007).Probability&statisticsfor
engineers&scientists(8ed.).UpperSaddleRiver,NJ:PearsonPrenticeHall.ISBN9780131877115.
24. Myers,RaymondH.Montgomery,DouglasC.Vining,G.GeoffreyRobinson,TimothyJ.(2010).
Generalizedlinearmodels:withapplicationsinengineeringandthesciences(2ed.).Hoboken,N.J:Wiley.
ISBN9780470454633.
25. Moore,DavidS.Notz,WilliamI.(2006).Statistics:conceptsandcontroversies(6thed.).NewYork:W.H.
Freeman.pp.Chapter7:Dataethics.ISBN9780716786368.
26. Ottoboni,M.Alice(1991).Thedosemakesthepoison:aplainlanguageguidetotoxicology(2nded.).New
York,N.Y:VanNostrandReinhold.ISBN0442006608.
27. Glantz,StantonA.(1992).Primerofbiostatistics(3rded.).ISBN0070235112.

References
Peirce,C.S.(18771878),"IllustrationsoftheLogicofScience"(series),PopularScience
Monthly,vols.1213.Relevantindividualpapers:
(1878March),"TheDoctrineofChances",PopularScienceMonthly,v.12,Marchissue,
pp.604(https://books.google.com/books?id=ZKMVAAAAYAAJ&jtp=604)615.Internet
ArchiveEprint(https://archive.org/stream/popscimonthly12yoummiss#page/612/mode/1up).
(1878April),"TheProbabilityofInduction",PopularScienceMonthly,v.12,pp.705
(https://books.google.com/books?id=ZKMVAAAAYAAJ&jtp=705)718.InternetArchive
Eprint(https://archive.org/stream/popscimonthly12yoummiss#page/715/mode/1up).
(1878June),"TheOrderofNature",PopularScienceMonthly,v.13,pp.203
(https://books.google.com/books?id=u8sWAQAAIAAJ&jtp=203)217.InternetArchive
Eprint(https://archive.org/stream/popularsciencemo13newy#page/203/mode/1up).
(1878August),"Deduction,Induction,andHypothesis",PopularScienceMonthly,v.13,
pp.470(https://books.google.com/books?id=u8sWAQAAIAAJ&jtp=470)482.Internet
ArchiveEprint(https://archive.org/stream/popularsciencemo13newy#page/470/mode/1up).
Peirce,C.S.(1883),"ATheoryofProbableInference",StudiesinLogic,pp.126181
(https://books.google.com/books?id=V7oIAAAAQAAJ&pg=PA126),Little,Brown,and
https://en.wikipedia.org/wiki/Design_of_experiments

9/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

Company.(Reprinted1983,JohnBenjaminsPublishingCompany,ISBN9027232717)

Furtherreading
Atkinson,A.C.(http://stats.lse.ac.uk/atkinson/)andDonev,A.N.
(http://www.maths.manchester.ac.uk/~adonev/)andTobias,R.D.
(http://support.sas.com/publishing/bbu/companion_site/index_author.html#tobias)(2007).
OptimumExperimentalDesigns,withSAS(http://books.google.se/books?id=oIHsrw6NBmoC).
OxfordUniversityPress
(http://www.us.oup.com/us/catalog/general/subject/Mathematics/ProbabilityStatistics/~~/dmlldz1
1c2EmY2k9OTc4MDE5OTI5NjYwNg==).pp.511+xvi.ISBN9780199296606.
Bailey,R.A.(2008).DesignofComparativeExperiments
(http://www.maths.qmul.ac.uk/~rab/DOEbook).CambridgeUniversityPress.ISBN9780521
683579.Prepublicationchaptersareavailableonline.
Box,G.E.P.,&Draper,N.R.(1987).Empiricalmodelbuildingandresponsesurfaces.New
York:Wiley.
Box,G.E.,Hunter,W.G.,Hunter,J.S.,Hunter,W.G.,"StatisticsforExperimenters:Design,
Innovation,andDiscovery",2ndEdition,Wiley,2005,ISBN0471718130
Caliski,TadeuszandKageyama,Sanpei(2000).Blockdesigns:ARandomizationapproach,
VolumeI:Analysis.LectureNotesinStatistics150.NewYork:SpringerVerlag.ISBN0387
985786.
GeorgeCasella(2008).Statisticaldesign
(http://www.springer.com/statistics/statistical+theory+and+methods/book/9780387759647).
Springer.ISBN9780387759654.
Ghosh,S.andRao,C.R.,ed.(1996).DesignandAnalysisofExperiments.HandbookofStatistics
13.NorthHolland.ISBN0444820612.
Goos,PeterandJones,Bradley(2011).OptimalDesignofExperiments:ACaseStudyApproach
(http://eu.wiley.com/WileyCDA/WileyTitle/productCd0470744618.html).Wiley.ISBN9780
470744611.
Hacking,Ian(September1988)."Telepathy:OriginsofRandomizationinExperimentalDesign".
Isis79(3):427451.doi:10.1086/354775(https://dx.doi.org/10.1086%2F354775).
JSTOR234674(https://www.jstor.org/stable/234674).MR1013489
(https://www.ams.org/mathscinetgetitem?mr=1013489).
Hinkelmann,KlausandKempthorne,Oscar(2008).DesignandAnalysisofExperiments.IandII
(Seconded.).Wiley(http://eu.wiley.com/WileyCDA/WileyTitle/productCd0470385510.html).
ISBN9780470385517.
Hinkelmann,KlausandKempthorne,Oscar(2008).DesignandAnalysisofExperiments,
VolumeI:IntroductiontoExperimentalDesign(https://books.google.com/?
id=T3wWj2kVYZgC&printsec=frontcover)(Seconded.).Wiley
(http://eu.wiley.com/WileyCDA/WileyTitle/productCd0471727563.html).ISBN9780
471727569.
Hinkelmann,KlausandKempthorne,Oscar(2005).DesignandAnalysisofExperiments,
Volume2:AdvancedExperimentalDesign(https://books.google.com/books?
id=GiYc5nRVKf8C)(Firsted.).Wiley
(http://eu.wiley.com/WileyCDA/WileyTitle/productCd0471551775.html).ISBN9780
471551775.
Mason,R.L.,Gunst,R.F.,&Hess,J.L.(1989).Statisticaldesignandanalysisofexperiments
withapplicationstoengineeringandscience.NewYork:Wiley.
Pearl,Judea.Causality:Models,ReasoningandInference,CambridgeUniversityPress,2000.
Peirce,C.S.(1876),"NoteontheTheoryoftheEconomyofResearch",AppendixNo.14in
CoastSurveyReport,pp.197201,NOAAPDFEprint
(http://docs.lib.noaa.gov/rescue/cgs/001_pdf/CSC0025.PDF#page=222).Reprinted1958in
CollectedPapersofCharlesSandersPeirce7,paragraphs139157andin1967inOperations
https://en.wikipedia.org/wiki/Design_of_experiments

10/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

Research(http://or.journal.informs.org/cgi/content/abstract/15/4/643)15(4):pp.643648,
abstractatJSTOR(http://www.jstor.org/stable/168276).Peirce,C.S.(1967)."NoteontheTheory
oftheEconomyofResearch".OperationsResearch15(4):643.doi:10.1287/opre.15.4.643
(https://dx.doi.org/10.1287%2Fopre.15.4.643).
Smith,Kirstine(1918)."OntheStandardDeviationsofAdjustedandInterpolatedValuesofan
ObservedPolynomialFunctionanditsConstantsandtheGuidanceTheyGiveTowardsaProper
ChoiceoftheDistributionoftheObservations".Biometrika12(1):185.doi:10.2307/2331929
(https://dx.doi.org/10.2307%2F2331929).
Taguchi,G.(1987).Jikkenkeikakuho(3rded.,VolI&II).Tokyo:Maruzen.Englishtranslation
editedbyD.Clausing.Systemofexperimentaldesign.NewYork:UNIPUB/KrausInternational.

Externallinks
Achapter(http://www.itl.nist.gov/div898/handbook/pri/section1/pri1.htm)froma
"NIST/SEMATECHHandbookonEngineeringStatistics"
(http://www.itl.nist.gov/div898/handbook/)atNIST
BoxBehnkendesigns(http://www.itl.nist.gov/div898/handbook/pri/section3/pri3362.htm)froma
"NIST/SEMATECHHandbookonEngineeringStatistics"
(http://www.itl.nist.gov/div898/handbook/)atNIST
ArticlesonDesignofExperiments(http://www.curiouscat.net/library/designofexperiments.cfm)
CaseStudiesandArticlesonDesignofExperiments(DOE)
(http://www.statease.com/articles.html)
Czitrom(1999)"OneFactorataTimeVersusDesignedExperiments",AmericanStatistician,53,
2.(http://www.questia.com/googleScholar.qst?docId=5001888588)
DesignResourcesServer(http://www.iasri.res.in/design)amobilelibraryonDesignof
Experiments.Theserverisdynamicinnatureandnewadditionswouldbepostedonthissitefrom
timetotime.
Gosset:AGeneralPurposeProgramforDesigningExperiments
(http://www.research.att.com/~njas/gosset/index.html)
SASExamplesforExperimentalDesign(http://www.wright.edu/~dvoss/book/DeanVoss.html)
MatlabSUrrogateMOdelingToolboxSUMOToolbox(http://sumowiki.intec.ugent.be)
MATLABcodeforDesignofExperiments+SequentialDesign+SurrogateModeling
DesignDB(http://web.cs.dal.ca/~peter/designdb/):Adatabaseofcombinatorial,statistical,
experimentalblockdesigns
TheIOptimalDesignAssistant
(http://obdoe.com/student/DOEResources/Assistant/assistantintro.php):afreeonlinelibraryofI
Optimaldesigns
WarningSignsinExperimentalDesignandInterpretation(http://norvig.com/experiment
design.html)byPeterNorvig,chiefofresearchatGoogle
KnowledgeBase,ResearchMethods(http://www.socialresearchmethods.net/kb/desexper.php):A
goodexplanationofthebasicideaofexperimentaldesigns
TheControlledExperimentvs.TheComparativeExperiment
(http://www.juliantrubin.com/fairguide/scientificmethod.html):"Howtoexperiment"forscience
fairprojects
Spall,J.C.(2010),"FactorialDesignforChoosingInputValuesinExperimentation:Generating
InformativeDataforSystemIdentification,"IEEEControlSystemsMagazine,vol.30(5),pp.38
53.(http://dx.doi.org/10.1109/MCS.2010.937677)Generalintroductionfromasystems
perspective
DOEusedforenginecalibrationreducesfuelconsumptionby2to4percent
(http://www.etas.com/en/products/ascmo.php)
Retrievedfrom"https://en.wikipedia.org/w/index.php?title=Design_of_experiments&oldid=681895489"
https://en.wikipedia.org/wiki/Design_of_experiments

11/12

9/21/2015

DesignofexperimentsWikipedia,thefreeencyclopedia

Categories: Designofexperiments Statisticalmethods Statisticaltheory Industrialengineering

Systemsengineering Qualitycontrol Quality Quantitativeresearch Engineeringstatistics
Experiments
Thispagewaslastmodifiedon20September2015,at07:02.
TextisavailableundertheCreativeCommonsAttributionShareAlikeLicenseadditionalterms
mayapply.Byusingthissite,youagreetotheTermsofUseandPrivacyPolicy.Wikipediaisa
registeredtrademarkoftheWikimediaFoundation,Inc.,anonprofitorganization.

https://en.wikipedia.org/wiki/Design_of_experiments

12/12