Diagnosis: Highlights

Cita Rosita Sigit Prakoeswa

Department of Dermato Venereology Dr Soetomo Hospital
Faculty of Medicine, Airlangga University, Surabaya
Tropical Disease Center, Airlangga University, Surabaya

What is diagnosis ?
Increase certainty about
presence/absence of
disease
Disease severity
Monitor clinical course
Assess prognosis
risk/stage within diagnosis
Plan treatment
Screening
Epidemiology
Knottnerus, BMJ 2002

By the end of this session,

you should be able to.
describe and illustrate key measures of
diagnostic test performance
represent diagnostic test performance

EBM Process
Drawing
Drawingconclusion
conclusion
That
Thatimpact
impacton
onpractice
practice
DOEs
DOEs
POEMs
POEMs

Patient
Encounter

Formulating the
Clinical Question

Appraising the
Evidence

Hierarchy
Hierarchyofofevidence
evidence
Pre
Preappraised
appraisedresources
resources

Diagnosis
Diagnosis
Therapy
Therapy
Prognosis
Prognosis
Etiology
Etiology

Searching the
Evidence

Patient
Patient
Intervention
Intervention
Comparison
Comparison
Outcome
Outcome

(Lang,
(Lang,2000)
2000)

What should I do
about this condition
or problem?
What cause
the problem?

CLINICAL
QUESTION

PROGNOSIS/RISK FACTORS

Does this person

have the condition
or problem?
Who will get
the condition
or problem?
How common
is the problem?
What are the
type of problem?

INTERVENTION

DIAGNOSIS

PROGNOSIS FACTORS

FREQUENCY & RATE

PHENOMENA / THOUGHTS
5

ACQ Diagnosis (PICO)

Patient /
Problem /
Population
In an otherwise
healthy 7-yearold boy with
sore throat

Intervention
(Index)

Comparison Outcome

how does the

clinical exam

compare to
throat culture

in diagnosing
GAS infection?

Controlled?
Randomized?

Longitudinal

Cross-sectional

nta
ime
r
e
exp

obse
rv

asio
nal

(+)
rol
t
n
Co

Con
trol
(-)

Research
Focus

Research
Approach

Research
Goal

Researcher
Involvement
7

Clinical Manifestation / Diagnosis / Prognosis / Therapy / Review

Hierarchy of study designs

Basic Principles (1)

Ideal diagnostic tests right answers:
(+) results in everyone with the disease and
( - ) results in everyone else
Usual clinical practice:
The test be studied in the same way it would
be used in the clinical setting
Observational study, and consists of:
Predictor variable (test result)
Outcome variable (presence / absence of the
disease)

Basic Principles (2)

Sensitivity, specificity
Prevalence, prior probability, predictive values
Likelihood ratios
Dichotomous scale, cutoff points (continuous
scale)
Positive (true and false), negative (true & false)
ROC (receiver operator characteristic) curve

What is the reason that there are

many parameters in diagnostic test?
Prevalence
Sensitivity (%)
Specificity (%)
Test
(+)
LR+
LRTest
PPV (%)
(-)
NPV (%)
Total
Pre-test Odds
Post-test Odds
Pre-test Probability (%)
Post-test Probability (%)

Disease Disease
(+)
(-)

Total

True pos
a

False
pos
b

a+b

False
neg
c

True neg
d

c+d

b+d

a+b+
c+d

a+c

METHOD 1:
NATURAL FREQUENCIES TREE
Populatio
n
1.000

IN EVERY 1.000 PEOPLE, 200 WILL HAVE THE DISEASE

Populatio
n
1.000
Disease +

Disease -

200

800

If these 1000 people are representative of the

population at risk, the assessed rate of those with the
disease (20%) represents the PREVALENCE of the
disease it can also be considered the PRE-TEST
PROBABILITY of having the disease

Sensitivity
The proportion of people who truly
have a designated disorder who are
so identified by the test.
Sensitive tests have few false
negatives.
When a test with a high Sensitivity is
Negative, it effectively rules out the
diagnosis of disease. SnNout

Sensitivity
Population

In other words, the

sensitivity is
190/200=95%

1.000
Disease +

Disease -

200

800

Test +

Test -

190

10

Test Alergi dengan Uji Kulit

Sensitivitas 95 %, artinya:
SnNout: bila hasil uji kulitnya (-): 95% out (dia bukan penderita alergi )

Specificity
The proportion of people who are
truly free of a designated disorder
who are so identified by the test.
Specific tests have few false positives
When a test is highly specific, a
positive result can rule in the
diagnosis. SpPin

Specificity
Population

In other words, the

specificity is 768/800
= 96%

1000
Disease +

Disease -

200

800

Test +

Test -

Test +

Test -

190

10

32

768

Test Alergi dengan Uji Kulit

Spesifitas 96 % artinya:
SpPin: bila hasil uji kulitnya (+): 96% in (dia penderita alergi)

Sensitivity & Specificity

FALSE
NEGATIVES

NON-CASES

CASES

Test cut-off

% of Group

FALSE
POSITIVES

NON-DISEASED
DISEASED
Positive

Negative
Degree of positivity on test

Numeric? (complex)

Sensitivity & Specificity

Sensitivity and Specificity are usually
considered properties of the test rather
than the setting, and are therefore usually
considered to remain constant.
However, sensitivity and specificity are
likely to be influenced by complexity of
differential diagnoses and a multitude of
other factors (spectrum bias).

Sensitivity & Specificity

Positive & Negative Predictive Value
For sensitivity and specificity, the
reference variable (denominator) is the
DISEASE
For predictive value, the reference
variable (denominator) is the TEST

Pre Test & Post Test Probability

Pre-test Probability
The probability of the target condition
being present before the results of a
diagnostic test are available. (prevalence)

Post-test Probability
The probability of the target condition
being present after the results of a
diagnostic test are available.
(Positive Predictive Value)

Positive Predictive Value

Population

This is also the POSTTEST PROBABILITY

of having the disease

1000

Disease +

Disease -

200

800

Test +

Test +

190

32

POSITIVE
PREDICTIVE
VALUE = 190/222
=86 %

Test -

Test -

10

768

Test Alergi dengan Uji Kulit

PPV 86 % artinya bila hasil uji kulitnya (+): kemungkinan dia
menderita alergi adalah 86%

Negative Predictive Value

Population
1000

Disease +

Disease -

200

800

Test +

Test +

190

32

NEGATIVE
PREDICTIVE
VALUE = 768/778
=99%

Test -

Test -

10

768

Test Alergi dengan Uji Kulit

NPV 99 % artinya bila hasil uji kulitnya (-): kemungkinan dia
tidak menderita alergi adalah 99 %

Positive & Negative

Predictive Value
The Positive Predictive Value of a test
will vary (according to the prevalence
of the condition in the chosen setting)

Predictive value & changing prevalence

Populatio
n
10.000
Disease +

Disease -

200

9.800

Prevalence reduced by an
order of magnitude from 20%
to 2%

Predictive value & changing prevalence

Populatio
n

Sensitivity
and Specificity
unchanged

10.000

Disease +

Disease -

200

9.800

Test +

Test +

190

392
Test -

Test -

10

9.408

Positive predictive value

at low prevalence
Populatio
n

Previously,
PPV was 86%

10.000

Disease +

Disease -

200

9.800

Test +

Test +

190

392

POSITIVE
PREDICTIVE
VALUE = 33%

Test -

Test -

10

9.408

Negative predictive value

at low prevalence
Populatio
n

Previously,
NPV was 99%

10.000

Disease +

Disease -

200

9.800

Test +

Test +

190

392

NEGATIVE
PREDICTIVE
VALUE >99%

Test -

Test -

10

9.408

Prediction of low prevalence events

Even highly specific tests, when applied
to low prevalence events, yield a high
number of false positive results
Because
of
this,
under
such
circumstances, the Positive Predictive
Value of a test is low
However, this has much less influence
on the Negative Predictive Value

Likelihood Ratio
Relative likelihood that a given test would be
expected in a patient with (as opposed to one
without) a disorder of interest.

LR=

probability (%) of a test result in patients with disease

probability (%) of the test result in patients without disease

Likelihood
Population
1000

Disease +
200
Test +
190
Test 10

The likelihood
that someone
with the disease
will have a
positive test is
190/200 or 95%
This is the same
as the sensitivity

Likelihood
Population
1000

Disease The likelihood that

someone without
the disease will
have a positive test
is 32/800 or 4%
This is the same as
the (1-specificity)

800
Test +
32
Test 768

Likelihood Ratio
LIKELIHOOD =
RATIO + (LR+)

LIKELIHOOD OF POSITIVE TEST

GIVEN THE DISEASE
LIKELIHOOD OF POSITIVE TEST
IN THE ABSENCE OF THE DISEASE

SENSITIVITY
1- SPECIFICITY

0.95
0.04

= 23.8

A Likelihood Ratio (LR) of 1.0

indicates an uninformative test (occurs when sensitivity and
specificity are both 50%)
The higher the Likelihood Ratio
the better the test (other factors being equal)

Test Alergi dengan Uji Kulit

LR+=23,8, artinya bila hasil uji kulitnya (+): hasil (+) ini dapat terjadi 23,8
kali lebih besar terjadi pada penderita alergi dibandingkan dengan yang
bukan penderita alergi

The diagonal line (representing

Sensitivity=0.5 and Specificity=0.5)
represents performance no better than
chance

Overall shape is
predicted by the
reciprocal relationship
between sensitivity
and specificity
The closer the curve
gets to Sensitivity=1
and Specificity=1, the
better the overall
performance of the
test
Hence the area under
the curve gives a
measure of the tests
performance

TRUE POSITIVE RATE (Sensitivity)

RECEIVER OPERATING CHARACTERISTIC CURVE

FALSE POSITIVE RATE (1Specificity)

AREA UNDER ROC CURVES

AREA=1.0

Sensitivity and specificity

both 100% - TEST PERFECT
Sensitivity and specificity
both 50% - TEST USELESS

AREA=0.5

The area under a

ROC curve will be
between 0.5 and 1.0

AREA UNDER ROC CURVES

Area = 0.7
(between 0.5 and
1.0)

Consider

(hypothetically) two patients drawn

randomly from the DISEASE+ and DISEASEgroups respectively

If

the test is used to guess which patient is

from the DISEASE+ group, it will be right 70%
of the time

RECEIVER OPERATING
CHARACTERISTIC (ROC) CURVE
This study compared
the performance of a
dementia screening
test in a community
sample (ACAT) and a
memory clinic
sample (MC)

Flicker L, Loguidice D,
Carlin JB, Ames D. The
predictive value of
dementia screening
instruments in clinical
populations. International
Journal of Geriatric
Psychiatry 1997 ; 12 : 203209

Diagnostic tests
Is not about finding absolute truth, but
about limiting uncertainty
establishes both the necessity and the
logical base for introducing probabilities,
pragmatic test-treatment thresholds ..\
Start thinking about
what youre going to do with the results of the
diagnostic test, and
whether doing the test will help your patients

Interpreting Diagnostic Studies

VIA - RaMMbo

Validity

Selection?

QUESTION:

Representative?

Participants

Index group (IG) &

Gold standard
Comparison Group (CG)

Outcome

VALIDITY

+
-

I
G

C
G

Reproducible
Maintain?

+ A
C

Measurements
blind subjective? OR
objective?

Diagnostic Accuracy Study:

Basic Design
Series of patients
Index test
Reference standard
Blinded cross-classification

Recruitment:
Was diagnostic test evaluated is representative
spectrum of patient?

Series of patients
Index test
Reference standard
Blinded cross-classification

Maintenance:
Was the endpoint of the reference standard
obtained for all subjects?

Series of patients
Index test
Reference standard
Blinded cross-classification

Measurement:
Were the assesors kept blind to the results of each
test and/or were the reference standard endpoint
objective

Series of patients
Index test
Reference standard
Blinded cross-classification

Spectrum Bias

Selected Patients
Index test
Reference standard
Blinded cross-classification

Verification Bias

Series of patients
Index test
Reference standard
Blinded cross-classification

Differential Reference Bias

Series of patients
Index test
Ref. Std A

Ref. Std. B

Blinded cross-classification

Observer Bias

Series of patients
Index test
Reference standard
Unblinded cross-classification

Importance

I
M
P
O
R
T
A
N
C
E

INTERVENTION

What should I do
about this condition
or problem?
What cause
the problem?

ETIOLOGY/RISK FACTORS

DIAGNOSIS

Does this person

have the condition
or problem?

Who will get

PROGNOSIS & PREDICTION
the condition
or problem?
FREQUENCY & RATE

How common
is the problem?
What are the
type of problem?

PHENOMENA / THOUGHTS

53

I
M
P
O
R
T
A
N
C
E

CLINICAL TRIAL

RRR, ARR, NNT

p & CI

PROGNOSIS

Survival curve
RR / OR
p & CI

DIAGNOSTIC

Sn,Sp,LH,PPV,NPV
p & CI
54

Applicability

PICO & Applicability

Applicability
Your question
(PICO)

Study

What do the
Result mean?

Importance
How well was
study done?

Validity

56

CRITICAL
APPRAISAL
DIAGNOSTIC
TEST

Critical appraisal diagnostic test

Use worksheet (VIA; RAMMbo)
STARD
Use supporting softwares
CAT Maker

Validity (1)
Apakah penelitian uji diagnostik dilakukan secara tersamar dengan baku
emas yang benar ?

Validity (2)
Apakah uji diagnostik dilakukan terhadap pasien dengan spektrum
penyakit atau kelainan yang memadai sehingga dapat diterapkan dalam
praktek sehari-hari?

Validity (3)
Apakah pemeriksaan dengan baku emas dilakukan tanpa memandang
hasil pemeriksaan dengan uji diagnostik ?

Important
Berapa Sn, Sp, LR+, LR-, PPV, NPV, Pre-test probability, Post-test
probability, Pre-test Odds, Post-test Odds ?

Applicable (1)
Apakah uji diagnostik tersebut tersedia, terjangkau dan akurat?

Applicable (2)
Apakah kita bisa memperkirakan pre-test probability (prevalens)
penyakit pada pasien kita ?

Applicable (3)
Apakah post-test probability yang dihitung akan mengubah tatalaksana
pasien kita?

Applicable (4)
Apakah secara keseluruhan uji diagnostik tersebut bermanfaat bagi
pasien ?

STARD initiative (25 items)

Standards for Reporting of Diagnostic Accuracy

Section and and

topic
Title, abstract, and
keywords
Introduction

Results
Participants
Test results
Estimates

Discussions

Methods
Participants
Test methods
Statistical methods
Bossuyt PM, Reitsma JB, Bruns, DE, Gatsonis CA, Glasziou PP et al. BMJ 2003,326:41-6

1st component of STARD

2nd component of STARD

Guides for deciding whether a screening or

early diagnostic maneuver does more good
than harm:

Does early diagnosis really lead to improved

survival, or quality of life, or both?
Are the early diagnosed patients willing partners in
the treatment strategy?
Is the time and energy it will take us to confirm the
diagnosis and provide (lifelong) care well spent?
Do the frequency and severity of the target disorder
warrant this degree of effort and expenditure?