Quasi-Experimental Designs in Psychology Research

Psychology 242 Introduction to Research
Quasi-experimental designs
Quasi-experiments
Experimental designs for studies in nature.
Studying naturally occurring events
Measurement studies
Retrospective designs Evaluate existing groups or program
Single shot survey or measure

Non-equivalent groups Time series designs
Psychology 242, Dr. McKirnan Research Ethics Back
Home page
Next
True v. quasi-experimental designs
True experiments:
Emphasize internal validity Assess cause & effect (in
relatively artificial environment)
Quasi-experiments:
Emphasize external validity Describe real / naturally occurring events Clear to exploratory hypotheses
Test clear, a priori hypotheses
Participants assigned to Existing or non-equivalent experimental v. control groups groups Random or matching Non-random assignment Participants & experimenter Participants not blind Blind to assignment Control group not possible?
Control study procedures Control often not possible Create / manipulate independent May not be able to manipulate variable the independent variable Control procedures & measures Partial control of procedures & measures
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs. Back
Home page
Next
Quasi-experiments: naturally occurring events
Studying naturally occurring events
Measurement studies Retrospective designs

Evaluate existing groups or program Single shot survey or measure Non-equivalent groups
Time series designs
Psychology 242, Dr. McKirnan
Research Ethics
Back
Home page
Next
Measurement studies
1. Naturally occurring events; examples

Event
(Predictor)
Study question
(Outcome)
Stress -> immune system Stress & coping
Natural disaster / stressor

3-mile island S.F. earthquake
Crime / trauma
Iraq service, 9 / 11 / 01 PTSD & treatment Contrails & climate change Voting patterns Health behavior
Back
Home page
Historical event
9/11 & air travel ban Economic collapse
Publicity / cultural event

Info. re: Hormone replacement
Week 12-13, quasi-experimental designs.
Next
Naturally occurring events, 2
Independent variable:
Predictor variable (e.g., natural disaster) often assessed after the event (post-hoc). Researcher has little control over dose / type of predictor
Participant selection

No control over who is exposure to event Some control over selection of sample (e.g., via targeted
sampling)
Many potential confounding variables Outcome (dependent) variables:

No control with archival data Some control with surveys Use retrospective (measured) variables to clarify interpretation of outcomes or test hypothesis.
Week 12-13, quasi-experimental designs. Back
Home page
Next
Naturally occurring events: Retrospective designs
Using retrospective (measured) variables to clarify interpretation of outcomes or test hypothesis.

Retrospective variable(s)
Social support Psych. history
[archive? Self-report?]
Event
(Predictor variable) earthquake [v. control city?] crime / trauma
[v. control people?]
Outcome variable
stress & coping mental health voting patterns
Personal attitudes
historical event
Demographics
cultural event
health behavior
Back
Home page
Next
Quasi-experiments: Existing groups
Studying naturally occurring events Measurement studies Retrospective designs
Evaluate existing groups or program
Single shot survey or measure

Non-equivalent groups
Time series designs
Home page
Next
Existing groups
Existing groups:
Single self-selected group; no comparison possible
users of psychotherapy (or any product) members of group or cult [contrast with demographically
matched controls?]
Two or more groups, with self-selection and / or "non-blind" assignment

Psychological interventions: therapy v. wait list, etc.
Two or more groups, no random assignment

Comparing schools / cities / existing groups
Home page
Next
One shot case studies

Naturally occurring event or social change Observe1
Group
May control selection of study group, or must use Convenience sample. Other data may be available about group.
Typically no control over event. Not a true Independent Variable
Dependent Variable(s): May or may not have control over measures (e.g., surveys v. archival measures).
Typical use: Surveys or measures after an event. Heuristic value: generating hypotheses for later study
or confirm controlled data in real world setting.

No control over selection of people into the event.
Internal / External validity:

Potentially no control over selection into measurement
group.
No control group; uncontrollable event, or other groups
may not need the intervention (e.g., therapy)

Back
Home page
Next
Example of one-shot case study
10
Example: Consumer Reports psychotherapy survey

[Click for paper]
Research questions: Does psychotherapy work from consumer view? Who gets therapy / what does it consist of? Do consumer responses vary by type of therapy? Research approach: One shot case study / survey Sampling frame: Any therapy or psychological service user No real information re: population of therapy users. Sampling procedure: 4,100 Consumer reports readers responding to in magazine mail-back survey form
Home page
Next
One shot: Consumer reports survey, 2
11
Experimental Controls
Evaluate by gender, type of treatment, medications, to provide more differentiated analysis
Negatives:
Selection bias no control over who got therapy (self-selection) of those who got therapy, no control over who
returned a survey (secondary self-selection) Cursory outcome measures: satisfaction rather than mental health
Positives:
Huge, national sample Wholly anonymous, 3rd party data collection; less bias Real world assessment of product quality
Home page
Next
One shot: Consumer reports survey, 3
12
Key distinction in psychological interventions: efficacy v. effectiveness research
Efficacy; true experimental design / Lab-basis

Rigorous controls; High internal validity
Test basic theory or highly specific technique
Do the specific ingredients (or theory) of this treatment validly induce the key outcome?
Effectiveness; quasi-experimental; natural or

applied setting
Less or no control; naturally occurring treatment High external validity
Does treatment work in real patients w/real therapists?

Home page
Next
Comparisons of study types:
13
(Consumer reports 4)
Efficacy experiment v.
One specific diagnosis Rigorous control group no treatment condition attention control Random assignment Manualized / uniform treatments High Fidelity to treatment method Fixed number of sessions. Well operationalized outcomes, e.g., clinician-diagnosed disorder Standard / validated self-report symptom scales
Effectiveness research
Multiple diagnoses & severities No control group, 2nd controls Archival, via pt. characteristics Self-selection; shopping Multiple / mixed treatments Highly tailored to patient # sessions is patient based. Diverse, self-referenced outcomes Subjective sense of wellness Lessening of problem behaviors or moods Personal assessment of functioning Self-rated: cannot be "blind" Diverse times since treatment Retrospective rather than prospective
Back
Home page
Blind" raters or diagnosticians

("single-blind: patient & therapist know what the treatment is..)
Patients followed for a fixed period

Next
Consumer reports survey, 5.
14
Survey findings on therapy effectiveness:

People who got more treatment (> 6 months) did better. For patients presenting For general health MH
problem(s) all specialists did about the same.
specialists did best, marriage counselors worst.
Back
Home page
Next
Consumer reports survey, 6.
15
Other effectiveness / descriptive findings:

3/4
went to mental-health specialist.
Patients who rated themselves worse at
outset made the most progress.

AA very highly evaluated Therapy did as well as medications
40% got drugs;
MDs gave medications to 83% of patients MH pros; 20% drug treatments 50% who got drugs got no counseling 20% got no information about side effects 40% of anti-anxiety drugs given > 1 year
Home page
Next
More complex case studies

Naturally occurring event Observation + Archival Controls
16
Group
Basic selection / convenience biases, uncontrollable event.
Dependent Variable(s): Combine survey or other measures with archival or ancillary data as Control variables.
Example 1: San Francisco earthquake & coping
Sampling frame: - Randomly selected survey participants

Outcomes: - Standardized mental health scales - Self-reports of stress
Quasi-controls: - population norms on outcomes - ancillary measures, e.g., social support

Findings:
- High rates of stress Rx, - Social support buffers stress

Home page
Next
One Shot + archival control: examples
17
Example 2: Stress and immune functioning

Event: - Three-mile Island nuclear accident
Sampling frame: - Randomly selected residents of geographic area around TMI Outcomes: - Blood draws for immune markers - Self-reports of stress
Quasi-controls: - Demographically matched sample - Archival data on health & illness
Findings:
- Long-term suppression of key immune markers (natural killer cells, T cells)

Home page
Next
One shot designs with archival controls, 3
18
Example 3: Psych. & Health effects of bereavement

Event: - Loss of spouse
Sampling frame: - Hospital records, self-selected spouses

Outcomes: - Blood draws for immune markers - Standardized mental health scales - Occupational functioning - Population norms on MH scales - Archival data: occupation & illness
Quasi-controls:
Findings:
- Long-term immune suppression - Social support buffered stress - impact of bereavement > other stressors
Home page
Next
One shot designs; Summary One Shot designs: no control over independent variable(s), only partial control over measurement:
19
An experiment is not possible
There cannot be a control group Pre- measures not possible or practical
Virtue:
Assess naturally occurring or uncontrollable socially or politically important events Provides real world look at processes that are typically studied in experiments: Effectiveness v. Efficacy data Archival data can help interpret the findings / control some alternate interpretations.
Liability:

lack of control group creates multiple threats to internal validity No pre-measure makes interpretation (e.g., of change) difficult.
Home page
Next
20
Evaluate existing groups or program Single shot survey or measure
Time series designs
Home page
Next
One group pre-test post-test

Observe1 Intervention or event Observe2 Outcome Assessment Typically controllable, but may be archival. Baseline Assessment May or may not have control over measures (e.g., surveys v. archival measures).
21
Group Selected or convenience sample.
Event or intervention May or may not be controllable by researcher, e.g., policy change.
Uses: Educational & social environments

Political or health policy change
Not feasible to have a control group System-wide intervention / social change (school, public health campaign..)
Home page
Next
Key design feature: no control group.

Group Observe1 Observe1 Intervention or event Observe2 Observe2
22
Confound
Threats to internal validity (confounds):

History
Maturation Reactive measures Statistical regression Mortality /
Historical / cultural events occur between baseline & follow-up.

Individual maturation or growth occurs between baseline & follow-up. People respond to being measured or being a measured a second time. Extreme scores at baseline regress to a more moderate level over time.
People leave the experiment nondrop-out randomly (i.e., for reasons that may affect the results). Week 12-13, quasi-experimental designs. Back Next
Home page
Examples: One group pre- post-, HIV testing
23
Example 1: Effects of HIV testing on sexual risk.

Event: - Receipt of HIV testing & counseling
Sampling frame: - Participants in testing centers Study structure: - Baseline retrospective interview at testing session - Follow-up interview 3 months later Quasi-controls: Outcomes:
- Population characteristics to predict between-group differences - Self-reports of sexual risk

Home page
Next
Example: One group pre- post-, HIV, 2
24
Effects of HIV testing on sexual risk, cont.
Findings:
- Significant shifts toward safety - Few demographic predictors of risk or risk change
Threats - Self-selection into testing group to internal - Mortality: non-random drop-out(?) validity - History: general shift in norms & behavior during study time may account for observed change - Instrument change; people may answer more conservatively during a follow-up interview
Home page
Next
Examples: One group pre- post- , Education
25
Example 2: Educational reform & No Child left behind testing requirements.

Intervention: - Standardized testing becomes integral to educational programs & school evaluation.
Sampling frame & - Longitudinal data across multiple Study structure: years in target school grades. - No control group possible.
Quasi-controls:
- Population characteristics to predict between-group differences

- Standardized test scores
Home page
Outcomes:
Next
Example: One group pre- post-, education, 2
Education reform & test scores.
26
Findings:

Modest, statistically significant increase in scores Usual demographic predictors of change; more affluent, better schools..
Internal validity?:
Reactive measures; teachers & students do better when measured; (they also cheat; see Houston Miracle article)
Instrumentation: kids get better at taking standardized tests, teacher better at teaching them History: General cultural shift

Education more prominent in city More affluent families sending kids to public schools
Home page
Next
One group pre- post- designs; Summary

An experiment is not possible There cannot be a control group Researchers have control over measurement and the independent variable
27
One group pre- post- test design useful where:

Virtues:

provide data on naturally occurring socially or politically important events Pre-measure allows researcher to interpret change & examine status of groups at baseline. History maturation statistical regression reactive measures mortality / drop-out
Back
Home page
Liability: lack of control

group creates multiple threats to internal validity:
Next
Non-equivalent two-group designs

Intervention or event
28
#1; Static Group Design

Group1 Group2 Observe1 Observe1
(No baseline)
Contrast group
Groups are not equivalent at baseline, due to.. Self-selection Non-random assignment Use of existing groups Participants not blind
Assessments may or may not be controlled Survey or interviews Archival / existing data, e.g., clinic records, grades Intervention or event may or may not be controlled by researcher; Existing program Experimental intervention Naturally occurring event (..9/11..)
Home page
Back
Next
Non-equivalent designs; pre- post-
29
#2 Two Group Pre- Post- Design

Group Group Observe1 Observe1 Intervention or event Contrast group Observe2 Observe2
Non-equivalent groups Self-selection Non-random assignment Use of existing groups Participants not blind
Intervention & Assessments often controlled by researcher in these designs.
Observation1 used to
Assess equivalence of groups at baseline Test for threats to internal validity:
Reactive measures History, mortality effects Regression effects
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Similar to true experimental design, except for non-equivalent groups
Back
Home page
Next
Examples: Non-equivalent groups, condoms
30
Example Non-equivalent control group design: Effects of condom distribution on sexual safety
Intervention: - Condom education & distribution in High School health classes
Sampling frame: - Schools in New York & Chicago - Schools matched for SES, race, size Study structure: - NY = intervention schools, Chicago are contrast schools. - Baseline, sexual health programming, end of year Follow-up Outcomes:
- Clinical measures: STDs - Self-reports: sexual activity & safety

Home page
Next
Examples: Non-equivalent groups, condoms, 2
31
Condom distribution, cont.
Findings:

NY (intervention) students; lower STD rate, safer sex NY and Chicago students; similar levels of sexual activity Thus; sexual health classes appeared to increase safety without increasing sexual activity. Reactive measures; Study is not blind; NY students know they are the intervention group
Internal validity?:
Non-equivalent groups: Possible differences between cities = unmeasured confounds

Home page
Next
Non-equivalent designs
Soloman 4-group design

Observe2 Observe2
32
Group 1
Observe1 Observe1
Intervention Contrast group
Group 2
Group 3 Group 4
Intervention
Contrast group
Observe2
Observe2
Groups 1 & 2:
Observation1 used to Assess equivalence of groups at baseline Test threats to internal validity
Groups 3 & 4:
Post-test only tests for reactive effects of assessment Compare 1+2 versus 3+4 Test interaction of treatment group x pre- post- versus post- only
Back
Home page
Next
Non-equivalent 2 group designs: Summary
33
Most common quasi-experimental approach. Used where:

Some form of control or contrast group is possible Groups cannot be equivalent:

Participants cannot be blind re: group assignment Random assignment not possible Must use existing or self-selected groups.
Virtue: Study natural / real world interventions Contrast group lessens major threats to internal validity Liability: non-equivalent groups = possible confound.

Home page
Next
34
Evaluate existing groups or program Single shot survey or measure
Time series designs

Home page
Next
Interrupted time series design

Measure1 M2 M3 M4 M5 M6 Intervention or event
35
Group
Test effect of intervention or event on ongoing series of measurements. Intervention may be experimental or observed
Policy shift, e.g., educational policy
Uncontrolled event; e.g., 9/11/01, Media event
Assessments may be experimental or archival

Successive cross-sectional surveys Traffic data, clinic or crime reports, test scores
Home page
Next
Time series designs
36
Group
Measure1
M2
M3
M4
M5
M6
Multiple baseline
Demonstrate highly stable effect long-term crime rates disease prevalence economic performance
Show steady rate of change
Hypothesis; tested by:
Shift in stable rate after intervention Increase / decrease in rate of change after intervention
Threats to internal validity:

sensitive to very local history Single group possibly prey to confound
Advantage for internal validity

Eliminates carryover effects of repeated measurement tests maturation, history, reactive measurement, etc

Home page
Next
Example of interrupted time series:
Shift in Baboon culture.
37
Core question: Do baboon troops develop and transmit a learned culture? Baseline: Long-term observational data on aggressiveness in a specific baboon troop. Intervention: Tuberculosis outbreak due to infected food. Dominant / aggressive males fed first
are selectively infected are naturally culled from troop Naturally occurring event in >20yr. ongoing
field study.
Back
Home page
Next
Baboon culture: findings
38
Quasi-controls: Parallel data from other baboon troops.

Outcome measures: Standardized indices of aggression & dominance behavior Core finding: With dominant males gone,
remaining males showed more cooperative behavior Enhanced cooperation was transmitted across generation, showing learned culture.
Home page
Next
Example: Interrupted time series data
39
The Magic Johnson effect on HIV testing

Data: Archival records of HIV tests reported to CDC,
collected monthly
Data show stable baseline over multiple observations Timing of intervention precise relative to data collection
Uncontrollable, naturally occurring event Tests hypothesis re: modeling effects in health behavior
Intervention: Magic reports infection on national TV.

Finding: Initial spike in testing rates, followed by

leveling off at higher base rate.
Initial increase expected Hypothesis tested by longer-term shift in base rate, available due to archival time-series data Effect found for both genders.
Home page
Next
Example of time-series data: Magic / HIV effect.
40
Time-series data showing shift in HIV testing after Magics announcement.
Magics Announcement
Initial spike New, higher base rate Low & variable baserate of testing
Multiple (monthly) measures.
Tesoriero, J.M., Sorin, M.D., Burrows, K.A., LaChance-McCullough, M.L. (1995). Harnessing the heightened public awareness of celebrity HIV disclosures: Magic and Cookie Johnson and HIV testing. AIDS Education and Prevention, 232-250.
Home page
Next
Multiple time series study
41
Multiple time series data

Group 1 Group 2 Measure1 Measure1 M2 M2 M3 M3 M4 M4 M5 M5 M6 M6
Groups typically formed by

blocking variable measured post-hoc; Health claims in NYC v. other cities post- 9/11/01 Younger v. older voting patterns post- Iraq invasion Heterosexual v. gay HIV testing rates post- Magic Johnson media event.
Hypothesis; tested by
interaction of blocking variable by repeated measure:
Is shift in stable rate ( rate of change) greater in one group than another?
Back
Home page
Next
Blocking variables
42
Testing blocking variables in the HIV testing time-series data. Core questions:
Both heterosexuals and Ethnic minorities had low HIV testing rates

May feel HIV is not relevant to them it is a white gay problem. They may lack resources or venues for testing.
Will having a prominent African-American Heterosexual disclose HIV+ status may change those perceptions? Heterosexuals will respond more strongly to the Magic Johnson media event than will gay/bisexual men.
African-American and Latino men and women will respond most strongly.
Home page
Hypotheses:
Next
Blocking variables: sexual orientation, 1.
43
Testing blocking variables: Gay / IDU data.

High base-line and high variability in testing rates among men with risky partners, and IDUs. Gay / bisexual men show less variable, but generally lower baserates. Risky men & IDUs slightly increase, with substantial variability.
Gay & bisexual men show no change.
Back
Home page
Next
Blocking variables: sexual orientation, 2.
44
Testing blocking variables: Heterosexuals.

In contrast to gay / bisexual men or IDUs, heterosexual show an initially low baserate. Followed by a large spike after the announcement
And a much higher new baserate.
The hypothesis that heterosexuals would be more affected by the Magic announcement was supported by the interaction of Time x the blocking variable of sexual orientation.
Home page
Next
Blocking variables: ethnic differences
45
Testing blocking variables: Ethnic differences.

African-Americans and Hispanics show low baserates and a high spike postannouncement
Both groups go back toward their baselines shortly post-announcement.
Back
Home page
Next
Blocking variables: ethnic differences, 2.
46
Ethnic differences: White participants.
HIV testing among Whites was similar to African-Americans & Hispanics at baseline, They showed stable, much higher testing rate after Magics HIV announcement.
Back
Home page
Next
Summary: Blocking variables in time series data
47
A series of measures before & after an event allows us to clearly identify patterns of behavior, and to test group differences (via blocking variables). The hypothesis that ethnic groups would differ was supported by interaction of Time x the blocking variable of ethnicity (but in a direction that was not predicted: Whites showed more change).
Home page
Back
Next
Time series designs: Summary
48
Time series is most common with archival data: existing, standard records collected for other purposes. Used where: The hypothesis concerns changes in long-term trends Typically an experiment cannot be run
Simple practicality or cost, e.g., health care issues Ethics; crime rates, rates of domestic violence, etc. The target events are not controllable.
Virtue:
Study natural / real world processes or interventions Blocking variables comparing time trends across groups -- lessens major threats to internal validity
Home page
Liability: lack of control = possible confound.

Next
Quick quiz
49
Researchers often use _____ to help interpret single shot surveys

A = paradigm change B = measurement studies C = experimental controls D = retrospective measures
Back
Home page
Next
Quick quiz, 2
50
Match:
A = Mortality / drop-out
Historical / cultural events occur between baseline & follow-up.

Individual maturation or growth occurs between baseline & follow-up. People respond to being measured or being a measured a second time. Extreme scores at baseline regress to a more moderate level over time.
B = Maturation
C = History D = Statistical regression E = Reactive measures
People leave the experiment nonrandomly (i.e., for reasons that may affect the results). Week 12-13, quasi-experimental designs. Back Next
Home page
Quick quiz, 2
51
Match:
A = Mortality / drop-out B = Maturation C = History
People respond to being measured or being a measured a second time.
D = Statistical regression E = Reactive measures

Home page
Next
Quick quiz, 3
52
Match:
B = Maturation
Growth or natural change between baseline & follow-up.
Back
Home page
Next
Quick quiz, 4
53
Match:
B = Maturation
Home page
People leave the experiment nonrandomly (i.e., for reasons that may affect the results).
Next
Quick quiz 5
54
Group
Measure1
M2
M3
M4
M5
M6
This is called a:
A = Threat to internal validity

B = Manipulation check C = Multiple baseline
D = ..lot of work.
Back
Home page
Next
Quasi-experiments; Summary
55
1. Study naturally occurring events that could not be brought into a lab or a true experiment.

Measurement studies
Retrospective designs
2. Evaluate existing groups or program(s)
Single shot survey or measure of an intervention
With or without control variables
Non-equivalent / pre-existing groups
Static group or 2 group pre- post- design
Time series designs, often with archival data
Trade off internal for external validity

Home page
Next
Exam issues
56
Key exam issues:

Slide 2: true v. quasi experiments Threats to internal validity Basic forms of quasi-experiments

Single shot Single group pre- post- test Non-equivalent two group designs:

Self-selection (in or out [mortality]) Existing groups Non-blind Non- random assignment
Interrupted time-series / group contrasts
Virtues (external validity) and problems (internal validity)

Home page
Next

Quasi-Experimental Designs in Psychology Research

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Quasi-Experimental Designs in Psychology Research

Uploaded by

Copyright:

Available Formats

Psychology 242 Introduction to Research

Experimental designs for studies in nature.

Studying naturally occurring events

Single shot survey or measure

Psychology 242 Introduction to Research

True v. quasi-experimental designs

Test clear, a priori hypotheses

Psychology 242 Introduction to Research

Quasi-experiments: naturally occurring events

Studying naturally occurring events

Measurement studies Retrospective designs

Time series designs

Psychology 242, Dr. McKirnan

Psychology 242 Introduction to Research

1. Naturally occurring events; examples

Natural disaster / stressor

Publicity / cultural event

Week 12-13, quasi-experimental designs.

Psychology 242 Introduction to Research

Naturally occurring events, 2

Many potential confounding variables Outcome (dependent) variables:

Psychology 242, Dr. McKirnan

Psychology 242 Introduction to Research

Naturally occurring events: Retrospective designs

Using retrospective (measured) variables to clarify interpretation of outcomes or test hypothesis.

Psychology 242, Dr. McKirnan

Week 12-13, quasi-experimental designs.

Psychology 242 Introduction to Research

Quasi-experiments: Existing groups

Studying naturally occurring events Measurement studies Retrospective designs

Evaluate existing groups or program

Single shot survey or measure

Psychology 242 Introduction to Research

Two or more groups, with self-selection and / or "non-blind" assignment

Two or more groups, no random assignment

Psychology 242 Introduction to Research

One shot case studies

Typically no control over event. Not a true Independent Variable

or confirm controlled data in real world setting.

Internal / External validity:

may not need the intervention (e.g., therapy)

Psychology 242, Dr. McKirnan

Psychology 242 Introduction to Research

Example of one-shot case study

Example: Consumer Reports psychotherapy survey

Psychology 242 Introduction to Research

One shot: Consumer reports survey, 2

Evaluate by gender, type of treatment, medications, to provide more differentiated analysis

Psychology 242 Introduction to Research

One shot: Consumer reports survey, 3

Key distinction in psychological interventions: efficacy v. effectiveness research

Efficacy; true experimental design / Lab-basis

Test basic theory or highly specific technique

Effectiveness; quasi-experimental; natural or

Does treatment work in real patients w/real therapists?

Psychology 242 Introduction to Research

Comparisons of study types:

Blind" raters or diagnosticians

Patients followed for a fixed period

Week 12-13, quasi-experimental designs.

Psychology 242 Introduction to Research

Consumer reports survey, 5.

Survey findings on therapy effectiveness: