Statistics 106

Winter 2013
Lecture Note 1
1

Analysis of Variance : an outline

1.1

Principles of experimental design

R.A. Fisher (1935), The Design of Experiments :

Replication: deals with variation/uncertainty, allows for generalization
Randomization: deals with confounding factors, allows for causal-effect statements
Blocking: reduces known but irrelevant sources of variation, improves efficiency
Examples of experiments
Want to study the effectiveness of a drug.
1. Experiment one : Apply the new drug to one patient and apply the old drug to another
patient
Patient with new drug recovered, and patient with old drug did not recover.
Conclusion: new drug is better ? Not necessarily. This can happen simply by chance.
2. Experiment two : 500 patients are chosen. 200 patients chose to try the new drug and 300
chose to try the old drug.
40% among patients using the new drug and 60% among patients using the old drug
recovered.
Conclusion: old drug is better ? Not necessarily. Patients opting for the new drug and
those opting for the old drug may be different to begin with.
3. Experiment three : 500 patients are chosen. 250 patients were randomly selected and
given the new drug and the rest 250 were given the old drug.
70% among patients using the new drug and 50% among patients using the old drug
recovered.
Conclusion: new drug is better ? Possibly. The empirical evidence of the difference in
efficacy seems significant.
Goal of experimental design
A controlled experiment aims to answer whether there is a cause-and-effect relationship
between the treatment and the response.
In Experiment three, randomization of treatments to experimental units occurs, while in
experiment two, randomization of treatments does not occur.
1

 Experiment three is a controlled experiment, while experiment two is more like an observational study
Observational study
The assignment of subjects into treatment groups is not determined by the investigator.
An observational study usually can only answer whether there is an association between the
treatment and the response. In general, external evidence is required to rule out possible
alternative explanations for a cause-and-effect relationship.
Controlled experiment vs. observational study
The reason that a cause-and-effect conclusion would be justified in a (well designed) controlled
experiment is that randomization tends to balance out the differences in other factors, leaving
the observed differences (if any) solely attributable to the treatment.
Why observational studies at all then ? It is sometimes impossible to do randomization, due
to ethnical reasons, practical feasibility, etc. (Think of diseases associated with hazardous
working conditions or lifestyles).
Examples: the study of effects of smoking on lung cancer incidence rate; the study of effects
of education on income.
ANOVA in a nutshell
ANOVA attributes observed variability to different factors; this is the major tool to study different
designs and analyze experimental data. ANOVA is also used for observational data.