Randomized Experimental

Design

What is an Experiment?
Campbell & Stanley stressed random
assignment to experimental treatments.
I stress manipulation of the independent
variable.
Quasi-Experiments: C&Ss term for
research where
there is a manipulated IV
but not random assignment to groups

Random Selection
Refers to the selection of data points from
a population into a random sample.
This selection procedure will be random if
each possible sample of size N is equally
likely to be sampled.
Random samples should be
representative of the population.
Our inferential statistics assume random
sampling.

Note. The tabled values are probabilities.

Y is random, X is not.
In Y, each time a single score is sampled, all
scores in the population are equally likely to be
sampled.

Random Assignment
Refers to the assignment of subjects to
treatment conditions.
Allows us to consider the populations
(subjects who will get special treatment
and those who will not) as equivalent prior
to treatment.
The samples will likely differ a little.

Two Basic Randomized Designs

Randomized Pretest-Posttest Control
Group Design
ROXO
ROO
Randomized Posttest Only Control Group
Design
RXO
RO

Noise Reducing Designs

These designs reduce noise (error
variance) and thus increase power.
treatment effect
Test statistic
noise
difference between means
t
standard error of difference
among groups variance
F
error variance

Randomized Blocks Designs

Matched pairs, randomized blocks.
Repeated measures or within-subjects.
Variance due to the blocking variable is
removed from error variance.

Blocks Error

Error
Treatment

Treatment

Blocks

Analysis of Covariance
Change a noise-producing extraneous
variable into a covariate that is included in
the statistical model.
Must be able to measure the covariate.
Variance due to covariate is removed from
the error variance.
Can have more than one covariate.

Factorial ANOVA
Convert a categorical extraneous variable
to an ANOVA factor.
Variance due to that factor will be removed
from the error term.
2x2FactorialDesign
RX11O
RX12O
RX21O
RX22O

Other Randomized Designs

Solomon Four Group Design
Controls threats to internal and
external validity as well as the
posttest only control group
design.
But has greater power.
And greater cost
Need more data
More complex analysis

ROXO
ROO
RXO
RO

Solomon Four Group Design

ANOVA
Arrange all four groups posttest scores
into a 2 x 2 ANOVA.
Pretested or Not x
Experimental Treatment or Not.
Significant Interaction Testing x
Treatment threat to External Validity
Main effect of pretesting.
Main effect of treatment

Treatment effect but no testing or

Testing x Treatment interaction

Pretest means in parentheses

Treatment and testing effects but

no Testing x Treatment interaction

Treatment and testing effects and a

Testing x Treatment interaction

Solomon Four Group Design

Pretest-Posttest Analysis
To gain power, analyze the pretestposttest portion of the design with
ANCOV, using pretest scores as covariate
Mixed factorial ANOVA
planned comparisons using t
control versus treatment change scores
(independent t)
pre versus post for control group (correlated t)
pre versus post for treatment group (correlated t)

Randomized SwitchingReplications Design

ROXOO
ROOXO
Attempt to control social threats to internal
validity.
Both groups get the special effect, one
early, one later.
May still be social effects with respect to
who gets it first.

Switching Replications
Temporary Treatment Effect

Group 1 got the treatment first.

Treatment is anxiety-reducing drug
DV = measure of anxiety reported by patient

Switching Replications
Persistent Treatment Effect

Treatment is psychotherapy
DV = measure of anxiety reported by patient
A third posttest could show the effect does not
last indefinitely.

Switching Replications
Continuing Treatment Effect

Treatment = cognitive psychotherapy

Anxiety continues to decline beyond the first
post-treatment observation, as patients get
better at employing the cognitive technique.