Professional Documents
Culture Documents
Problem of criterion
Perceptual decisions are influenced by many factors:
-! -! Strength of signal relative to background level of noise Persons sensitivity to the signal Relative frequency of occurrence of the signal Persons level of motivation Costs associated with hits and false alarms Personal biases (individual differences)
signal + noise
S response
S response
S response
S response
S response
S response
signal + noise
S response
S response
S response
S response
S response
S response
abnormal
incorrect
correct
Hit correct diagnosis of disease Miss physician tells patient that no disease exists when one does False alarm unnecessary operation or treatment Correct rejection patient properly diagnosed with no disease
abnormal
incorrect
correct
Hit correct diagnosis of disease Miss physician tells patient that no disease exists when one does False alarm unnecessary operation or treatment Correct rejection patient properly diagnosed with no disease Which error is more costly?!
tumor Miss
abnormal
False alarm
Hit
Accused is" Accused is" innocent guilty Jury decision innocent correct verdict mistake mistake correct verdict
guilty
P?
R PR R R R R R R R R
noise
signal + noise
noise
signal + noise
criterion
Signal present
criterion miss hit
no
yes
Signal absent
criterion correct rejection false alarm
no
yes
Balanced: false alarm and miss rates are equal Liberal: the observer says yes whenever there may be a signal Conservative: decision is yes only when it is almost certain that there is a signal
liberal
criterion
balanced
criterion
criterion
conservative
Discriminability: how well the observer can separate the presence of signal from its absence - overlap between the two distributions - Measured by d (discriminability index, also called sensitivity) - computed as: d" = z(Hit) z(FA)
criterion
Liberal criterion
Tumor present
CR M
FA
Conservative criterion
Tumor present
M CR CR M FA H
hit rate
100%
criterion
criterion
criterion
criterion
criterion
criterion
criterion
ROC Curve"
100" Hit rate (%)" 80" 60" 40" 20" 0" 0" 20" 40" 60" 80" 100" False alarm rate (%)"
ROC Curve"
100" Hit rate (%)" 80" 60" D! 40" 20" E! A! C! B!
! Which observer did best?" ! Who was just there completing the subject pool requirement?"
0" 0" 20" 40" 60" 80" 100" False alarm rate (%)"
http://www-psych.stanford.edu/~lera/psych115s/notes/signal/ http://psych.hanover.edu/JavaTest/Media/Chapter02.html Interactive Model 2.x: Signal Detection Theory Interactive Model 2.x: Signal Detection Illustration Interactive Model 2.x: Decisions In SDT Interactive Model 2.x: Receiver Operating Characteristic
! Sensitivity (d) depends on ! signal strength ! noise strength ! observer sensitivity ! Criterion depends on ! Personal bias ! Cost/benefit factors (risk factors) ! Signal frequency
! Version #2 - Payoffs ! The other half will do one of the experiments manipulating sensitivity ! Version #4 Manipulation of the intensity of noise ! Cover general SDT issues in intro ! Compute C and d (detailed instruction in SG: pages 1.11 - 1.12) ! Hand plot YOUR data on the empty ROC curve ! use empty symbols for criterion exp (2X, label each, connect by line) ! use full symbols for sensitivity exp (2X, label each, connect by line) ! Group data will be emailed
! Half of the class will do one of the experiments manipulating the criterion