Professional Documents
Culture Documents
Methodology
• Introduction
• Hypothesis testing
• ROC curve
• Area under the ROC curve (AUC)
• Examples using ROC
• Concluding remarks
Introduction to ROC curves
Test Result
Threshold
Call these patients “negative” Call these patients “positive”
Test Result
Some definitions ...
Call these patients “negative” Call these patients “positive”
True Positives
Test Result
True
negatives
Test Result
False
negatives
Test Result
‘‘-’’ ‘‘+’’
Test Result
‘‘-’’ ‘‘+’’
Test Result
0%
0% 100%
False Positive Rate
(1-specificity)
ROC curve comparison
100% 100%
True Positive Rate
100% 100%
True Positive Rate
True Positive
Rate
0
0 %
% 0 100
0 100 False Positive %
False Positive % %
% Rate
Rate
• Binormal model:
– X ~ N(0,1) in nondiseased population
– X ~ N(a, 1/b) in diseased population
• Then
ROC(t) = Φ (a + bΦ -1(t)) for 0 < t < 1
• Estimate a, b by ML using readings from
sets of diseased and nondiseased patients
ROC curve estimation with
continuous data
• Many biochemical measurements are in fact
continuous, e.g. blood glucose vs. diabetes
• Can also do ROC analysis for continuous (rather
than binary or ordinal) data
• Estimate ROC curve (and smooth) based on
empirical ‘survivor’ function (1 – cdf) in
diseased and nondiseased groups
• Can also do regression modeling of the test
result
• Another approach is to model the ROC curve
directlyas a function of covariates
Area under ROC curve (AUC)
AUC = 100%
True Positive Rate
True Positive
Rate
AUC = 50%
0
0 %
% 0 100
0 100 False Positive %
False Positive % %
% Rate
Rate
100% 100%
AUC = 90%
True Positive
Rate
0 0
% %
0 100 100
False Positive 0
% % False Positive %
%
Rate Rate
Interpretation of AUC